Re: [go-nuts] Get a copy of reflect.Value's underlying value

[Josh Humphries]

On Tue, Aug 1, 2017 at 2:43 PM, roger peppe  wrote:

> On 1 August 2017 at 19:33, Josh Humphries  wrote:
> > On Tue, Aug 1, 2017 at 11:44 AM, roger peppe  wrote:
> >>
> >> On 1 August 2017 at 13:57, Josh Humphries 
> wrote:
> >> > Although that solution creates a heap-allocated tmp for every element
> in
> >> > the
> >> > slice. Using an interface, the value will be inlined instead of
> >> > heap-allocated if it fits without boxing (so most primitive types and
> >> > pointer types won't need heap allocation).
> >>
> >> As it happens, it's the other way round - the solution using Interface
> can
> >> make an allocation per element, but the solution using Value does not.
> >
> >
> > The solution using Value is calling reflect.New inside the loop. Why do
> you
> > say that it does not make an allocation per element?
>
> The code I posted doesn't call reflect.New inside the loop - perhaps
> you're looking at some different code? (for the record, I'm talking
> about line 45 of https://play.golang.org/p/Q0VHbfL7Ij)
>

Ah, I see. Thanks for the clarification. That makes sense to allocate just
one for the whole operation.


>
> > Also, I'm not sure I follow that using Interface makes an allocation per
> > element. My understanding is that the actual value can be stored in the
> > interface value if it is small enough (1 or 2 words?). So boxing
> primitives
> > and pointers should generally store the value in the interface tuple
> (which
> > itself is stack allocated) as opposed to allocating heap space for it and
> > storing a pointer.
>
> As I think was pointed out earlier in this thread, this has changed since
> Go 1. Only pointer values are stored in interface values now - non-pointer
> value (for example the ints in the example) will usually cause an
> allocation.
> There are some optimisations for some cases that avoid that: creation
> of an interface from a constant value, and creation of an interface from
> small numeric values.
>
>
Ugh. I did not realize that. I had not dug into it too much other than
reading a Go blog post and this article by Russ Cox
 (which suggest this memory
optimization is done).

I am guessing the conditional that decides between inline value vs.
pointer-chasing was too much runtime overhead?


> Try running the benchmark code that I posted. I think you'll find
> that the code that calls Interface is slower and allocates more,
> because it has to allocate a storage slot every time it creates
> an interface from an int, which the solution using Value does not,
> because it allocates the temporary only once for a given call to Reverse
> (even that could be amortised by using sync.Pool, but it's almost
> certainly not worth it).
>
>   cheers,
> rog.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Get a copy of reflect.Value's underlying value

[roger peppe]

On 1 August 2017 at 19:33, Josh Humphries  wrote:
> On Tue, Aug 1, 2017 at 11:44 AM, roger peppe  wrote:
>>
>> On 1 August 2017 at 13:57, Josh Humphries  wrote:
>> > Although that solution creates a heap-allocated tmp for every element in
>> > the
>> > slice. Using an interface, the value will be inlined instead of
>> > heap-allocated if it fits without boxing (so most primitive types and
>> > pointer types won't need heap allocation).
>>
>> As it happens, it's the other way round - the solution using Interface can
>> make an allocation per element, but the solution using Value does not.
>
>
> The solution using Value is calling reflect.New inside the loop. Why do you
> say that it does not make an allocation per element?

The code I posted doesn't call reflect.New inside the loop - perhaps
you're looking at some different code? (for the record, I'm talking
about line 45 of https://play.golang.org/p/Q0VHbfL7Ij)

> Also, I'm not sure I follow that using Interface makes an allocation per
> element. My understanding is that the actual value can be stored in the
> interface value if it is small enough (1 or 2 words?). So boxing primitives
> and pointers should generally store the value in the interface tuple (which
> itself is stack allocated) as opposed to allocating heap space for it and
> storing a pointer.

As I think was pointed out earlier in this thread, this has changed since
Go 1. Only pointer values are stored in interface values now - non-pointer
value (for example the ints in the example) will usually cause an allocation.
There are some optimisations for some cases that avoid that: creation
of an interface from a constant value, and creation of an interface from
small numeric values.

Try running the benchmark code that I posted. I think you'll find
that the code that calls Interface is slower and allocates more,
because it has to allocate a storage slot every time it creates
an interface from an int, which the solution using Value does not,
because it allocates the temporary only once for a given call to Reverse
(even that could be amortised by using sync.Pool, but it's almost
certainly not worth it).

  cheers,
rog.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Get a copy of reflect.Value's underlying value

[Josh Humphries]

On Tue, Aug 1, 2017 at 11:44 AM, roger peppe  wrote:

> On 1 August 2017 at 13:57, Josh Humphries  wrote:
> > Although that solution creates a heap-allocated tmp for every element in
> the
> > slice. Using an interface, the value will be inlined instead of
> > heap-allocated if it fits without boxing (so most primitive types and
> > pointer types won't need heap allocation).
>
> As it happens, it's the other way round - the solution using Interface can
> make an allocation per element, but the solution using Value does not.
>

The solution using Value is calling reflect.New inside the loop. Why do you
say that it does not make an allocation per element?

Also, I'm not sure I follow that using Interface makes an allocation per
element. My understanding is that the actual value can be stored in the
interface value if it is small enough (1 or 2 words?). So boxing primitives
and pointers should generally store the value in the interface tuple (which
itself is stack allocated) as opposed to allocating heap space for it and
storing a pointer.

For larger types (e.g. structs beyond a certain size) it will indeed
allocate on the heap. I don't see any simple way around this. There may be
some clever tricks, possibly involving cgo or asm, to avoid it though
(maybe like how reflect.Swapper
 works, although it
also heap allocates for values >8 bytes).



>
> https://play.golang.org/p/Q0VHbfL7Ij
>
> Additionally, the solution using Interface can lose the type information
> in some cases and panic as a result.
>


A simple guard for the nil case is necessary to prevent the panic (since
the nil interface results in an invalid reflect.Value):

tmp := a.Interface()
var tmprv reflect.Value
if tmp == nil {
  tmprv = reflect.Zero(a.Type())
} else {
  tmprv = reflect.ValueOf(tmp)
}
a.Set(b)
b.Set(tmprv)


>
> https://play.golang.org/p/WocF9CaPoR
>
>   cheers,
> rog.
>
>
> >
> > 
> > Josh Humphries
> > jh...@bluegosling.com
> >
> > On Tue, Aug 1, 2017 at 7:29 AM, roger peppe  wrote:
> >>
> >> FWIW, you don't have to use Interface to do the swap:
> >>
> >> https://play.golang.org/p/O8lGJGGOXP
> >>
> >> On 31 July 2017 at 15:18, eZio Pan  wrote:
> >> > Hello,
> >> > I want to build a "universal slice reverser" with reflect.MakeFunc.
> But
> >> > I
> >> > don't know how to get a copy of reflect.Value's underlying value,
> which
> >> > make
> >> > result not correct.
> >> >
> >> > Here is the code:
> >> >
> >> > package main
> >> >
> >> > import (
> >> > "fmt"
> >> > "reflect"
> >> > )
> >> >
> >> > func reverse(in []reflect.Value) (out []reflect.Value) {
> >> > inls := in[0]
> >> > inlslen := inls.Len()
> >> >
> >> > for i, j := 0, inlslen-1; i < j; i, j = i+1, j-1 {
> >> > a := inls.Index(i)
> >> > b := inls.Index(j)
> >> > // how to get underlying value of a and b ?
> >> > a.Set(b)
> >> > b.Set(a)
> >> > }
> >> > return in
> >> > }
> >> >
> >> > var intFlipper func([]int) []int
> >> >
> >> > func main() {
> >> > emptyFunc := reflect.ValueOf().Elem()
> >> > pseudoFunc := reflect.MakeFunc(emptyFunc.Type(), reverse)
> >> > emptyFunc.Set(pseudoFunc)
> >> > fmt.Printf("%v\n", intFlipper([]int{2, 3, 4, 5, 6, 7}))
> >> > }
> >> >
> >> > My code return [7 6 5 5 6 7], but excepting result is [7 6 5 4 3 2]
> >> >
> >> > --
> >> > You received this message because you are subscribed to the Google
> >> > Groups
> >> > "golang-nuts" group.
> >> > To unsubscribe from this group and stop receiving emails from it, send
> >> > an
> >> > email to golang-nuts+unsubscr...@googlegroups.com.
> >> > For more options, visit https://groups.google.com/d/optout.
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "golang-nuts" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an
> >> email to golang-nuts+unsubscr...@googlegroups.com.
> >> For more options, visit https://groups.google.com/d/optout.
> >
> >
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Get a copy of reflect.Value's underlying value

[roger peppe]

On 1 August 2017 at 13:57, Josh Humphries  wrote:
> Although that solution creates a heap-allocated tmp for every element in the
> slice. Using an interface, the value will be inlined instead of
> heap-allocated if it fits without boxing (so most primitive types and
> pointer types won't need heap allocation).

As it happens, it's the other way round - the solution using Interface can
make an allocation per element, but the solution using Value does not.

https://play.golang.org/p/Q0VHbfL7Ij

Additionally, the solution using Interface can lose the type information
in some cases and panic as a result.

https://play.golang.org/p/WocF9CaPoR

  cheers,
rog.


>
> 
> Josh Humphries
> jh...@bluegosling.com
>
> On Tue, Aug 1, 2017 at 7:29 AM, roger peppe  wrote:
>>
>> FWIW, you don't have to use Interface to do the swap:
>>
>> https://play.golang.org/p/O8lGJGGOXP
>>
>> On 31 July 2017 at 15:18, eZio Pan  wrote:
>> > Hello,
>> > I want to build a "universal slice reverser" with reflect.MakeFunc. But
>> > I
>> > don't know how to get a copy of reflect.Value's underlying value, which
>> > make
>> > result not correct.
>> >
>> > Here is the code:
>> >
>> > package main
>> >
>> > import (
>> > "fmt"
>> > "reflect"
>> > )
>> >
>> > func reverse(in []reflect.Value) (out []reflect.Value) {
>> > inls := in[0]
>> > inlslen := inls.Len()
>> >
>> > for i, j := 0, inlslen-1; i < j; i, j = i+1, j-1 {
>> > a := inls.Index(i)
>> > b := inls.Index(j)
>> > // how to get underlying value of a and b ?
>> > a.Set(b)
>> > b.Set(a)
>> > }
>> > return in
>> > }
>> >
>> > var intFlipper func([]int) []int
>> >
>> > func main() {
>> > emptyFunc := reflect.ValueOf().Elem()
>> > pseudoFunc := reflect.MakeFunc(emptyFunc.Type(), reverse)
>> > emptyFunc.Set(pseudoFunc)
>> > fmt.Printf("%v\n", intFlipper([]int{2, 3, 4, 5, 6, 7}))
>> > }
>> >
>> > My code return [7 6 5 5 6 7], but excepting result is [7 6 5 4 3 2]
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to golang-nuts+unsubscr...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Get a copy of reflect.Value's underlying value

[Jan Mercl]

On Tue, Aug 1, 2017 at 2:58 PM Josh Humphries  wrote:

> Using an interface, the value will be inlined instead of heap-allocated
if it fits without boxing (so most primitive types and pointer types won't
need heap allocation).

This was true some years ago, but it is no more.

-- 

-j

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Get a copy of reflect.Value's underlying value

[Josh Humphries]

Although that solution creates a heap-allocated tmp for every element in
the slice. Using an interface, the value will be inlined instead of
heap-allocated if it fits without boxing (so most primitive types and
pointer types won't need heap allocation).


*Josh Humphries*
jh...@bluegosling.com

On Tue, Aug 1, 2017 at 7:29 AM, roger peppe  wrote:

> FWIW, you don't have to use Interface to do the swap:
>
> https://play.golang.org/p/O8lGJGGOXP
>
> On 31 July 2017 at 15:18, eZio Pan  wrote:
> > Hello,
> > I want to build a "universal slice reverser" with reflect.MakeFunc. But I
> > don't know how to get a copy of reflect.Value's underlying value, which
> make
> > result not correct.
> >
> > Here is the code:
> >
> > package main
> >
> > import (
> > "fmt"
> > "reflect"
> > )
> >
> > func reverse(in []reflect.Value) (out []reflect.Value) {
> > inls := in[0]
> > inlslen := inls.Len()
> >
> > for i, j := 0, inlslen-1; i < j; i, j = i+1, j-1 {
> > a := inls.Index(i)
> > b := inls.Index(j)
> > // how to get underlying value of a and b ?
> > a.Set(b)
> > b.Set(a)
> > }
> > return in
> > }
> >
> > var intFlipper func([]int) []int
> >
> > func main() {
> > emptyFunc := reflect.ValueOf().Elem()
> > pseudoFunc := reflect.MakeFunc(emptyFunc.Type(), reverse)
> > emptyFunc.Set(pseudoFunc)
> > fmt.Printf("%v\n", intFlipper([]int{2, 3, 4, 5, 6, 7}))
> > }
> >
> > My code return [7 6 5 5 6 7], but excepting result is [7 6 5 4 3 2]
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to golang-nuts+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Get a copy of reflect.Value's underlying value

[roger peppe]

FWIW, you don't have to use Interface to do the swap:

https://play.golang.org/p/O8lGJGGOXP

On 31 July 2017 at 15:18, eZio Pan  wrote:
> Hello,
> I want to build a "universal slice reverser" with reflect.MakeFunc. But I
> don't know how to get a copy of reflect.Value's underlying value, which make
> result not correct.
>
> Here is the code:
>
> package main
>
> import (
> "fmt"
> "reflect"
> )
>
> func reverse(in []reflect.Value) (out []reflect.Value) {
> inls := in[0]
> inlslen := inls.Len()
>
> for i, j := 0, inlslen-1; i < j; i, j = i+1, j-1 {
> a := inls.Index(i)
> b := inls.Index(j)
> // how to get underlying value of a and b ?
> a.Set(b)
> b.Set(a)
> }
> return in
> }
>
> var intFlipper func([]int) []int
>
> func main() {
> emptyFunc := reflect.ValueOf().Elem()
> pseudoFunc := reflect.MakeFunc(emptyFunc.Type(), reverse)
> emptyFunc.Set(pseudoFunc)
> fmt.Printf("%v\n", intFlipper([]int{2, 3, 4, 5, 6, 7}))
> }
>
> My code return [7 6 5 5 6 7], but excepting result is [7 6 5 4 3 2]
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Get a copy of reflect.Value's underlying value

[Josh Humphries]

You have to use a temporary to swap the values:

tmp := a.Interface()
a.Set(b)
b.Set(reflect.ValueOf(tmp))



*Josh Humphries*
jh...@bluegosling.com

On Mon, Jul 31, 2017 at 10:18 AM, eZio Pan  wrote:

> Hello,
> I want to build a "universal slice reverser" with reflect.MakeFunc. But I
> don't know how to get a copy of reflect.Value's underlying value, which
> make result not correct.
>
> Here is the code:
>
> package main
>
> import (
> "fmt"
> "reflect"
> )
>
> func reverse(in []reflect.Value) (out []reflect.Value) {
> inls := in[0]
> inlslen := inls.Len()
>
> for i, j := 0, inlslen-1; i < j; i, j = i+1, j-1 {
> a := inls.Index(i)
> b := inls.Index(j)
> // how to get underlying value of a and b ?
> a.Set(b)
> b.Set(a)
> }
> return in
> }
>
> var intFlipper func([]int) []int
>
> func main() {
> emptyFunc := reflect.ValueOf().Elem()
> pseudoFunc := reflect.MakeFunc(emptyFunc.Type(), reverse)
> emptyFunc.Set(pseudoFunc)
> fmt.Printf("%v\n", intFlipper([]int{2, 3, 4, 5, 6, 7}))
> }
>
> My code return [7 6 5 5 6 7], but excepting result is [7 6 5 4 3 2]
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Get a copy of reflect.Value's underlying value

[Jan Mercl]

On Mon, Jul 31, 2017 at 4:18 PM eZio Pan  wrote:

> My code return [7 6 5 5 6 7], but excepting result is [7 6 5 4 3 2]

https://play.golang.org/p/yraJZFDjS3

-- 

-j

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.