Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

[JX Zhang]

It seems you are right.
I'm not sure if my understanding is correct:
I guess that this problem occured because gollvm just exposes the interface 
of exported functions in the lib, so the private function can't be accessed.
I  want to know if there is any way to make the internal functions of the 
standard library accessible through linkname？

Hi,
>
> *>Thanks for your attention, but I tried to write a demo with go:linkname.*
> *>In fact, it works well with gollvm...So maybe it is not the exact cause 
> for the problem*
>
> The problem is not that gollvm fails to implement go:linkname-- the 
> problem is the way that this package (http://github.com/ugorji/go)  is 
> using go:linkname.
>
> The Go runtime and standard library has a public, exported API; this sort 
> of use of go:linkname to reach into the internals of a standard package (in 
> this case, calling an unexported function) is not supported.
>
> Folks who write such code should understand that it can break at any time 
> from release to release or across Go implementations.
>
> Than
>
>
>
>
> On Mon, Mar 29, 2021 at 3:35 AM JX Zhang  wrote:
>
>> >  go:linkname to refer directly to functions that are defined but not 
>> exported by the standard library. 
>> > This is not supported and is likely to break with any new release. It 
>> evidently breaks with GoLLVM.
>>
>> Thanks for your attention, but I tried to write a demo with go:linkname.
>> In fact, it works well with gollvm...So maybe it is not the exact cause 
>> for the problem
>>
>> Herei is my code:
>>
>> hello/hello.go
>> ```
>> package hello
>>
>> import (
>> "fmt"
>> _ "unsafe"
>> )
>> //go:linkname hellox hello.hellox
>> func hellox(x string) {
>> fmt.Println(x)
>> }
>> ```
>>
>> main.go
>> ```
>> package main
>>
>> import (
>> _ "mypackage/hello"
>> _ "unsafe"
>> )
>>
>> //go:linkname hel hello.hellox
>> func hel(x string)
>>
>> func main() {
>> hel("aaa")
>> //println("")
>> }
>> ```
>>
>> 在2021年3月29日星期一 UTC+8 上午12:37:49 写道：
>>
>>> On Sun, Mar 28, 2021 at 9:28 AM 张嘉熙  wrote: 
>>> > 
>>> > For s2-geojson, I meet: 
>>> > ``` 
>>> > # github.com/pantrif/s2-geojson/cmd/s2-geojson 
>>> > 
>>> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>>>  
>>> github_0com_1ugorji_1go_1codec.intf2impls.intf2impl: error: undefined 
>>> reference to 'reflect.unsafe_New' 
>>> > 
>>> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>>>  
>>> github_0com_1ugorji_1go_1codec.Decoder.interfaceExtConvertAndDecode: error: 
>>> undefined reference to 'reflect.unsafe_New' 
>>> > decode.go:509: error: undefined reference to 'reflect.unsafe_New' 
>>> > decode.go:14935: error: undefined reference to 'reflect.unsafe_New' 
>>>
>>> This is a problem with github.com/ugorji/go. The file 
>>> code/helper_unsafe.go uses go:linkname to refer directly to functions 
>>> that are defined but not exported by the standard library. This is 
>>> not supported and is likely to break with any new release. It 
>>> evidently breaks with GoLLVM. 
>>>
>>> Ian 
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/5d763cdf-1147-4abe-aea7-9f6efa39e930n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/57828e17-e21d-4167-9f33-0694bfde14b3n%40googlegroups.com.

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

['Ugorji Nwoke' via golang-nuts]

Than,

Appreciate you saying that. And you're very welcome.

On Tue, Mar 30, 2021 at 8:29 PM Than McIntosh  wrote:

> Thank you @Ugorji Nwoke   , I appreciate your going
> the extra mile to include gollvm in your testing.
>
> Cheers, Than
>
>
> On Tue, Mar 30, 2021 at 11:17 AM Ugorji Nwoke  wrote:
>
>> Ugorji here - author of the github.com/ugorji/go/codec
>>  package.
>>
>> The notes below are for folks that are interested in why we use unsafe,
>> and how we mitigate concerns around it.
>>
>> As Peter mentioned above, you can pass the build tag "codec.safe" to
>> bypass using unsafe code where we try to reach deeper into the runtime
>> implementation to optimize.
>>
>> I have been supporting gccgo for a while now, but never tested with
>> gollvm, as it has not been released at this time. I built a gollvm version
>> yesterday (from source) and tested it, and made some changes (see here
>> 
>> and here
>> )
>> so that gollvm will work in the default (high-performance) mode. I plan to
>> cut a new release v1.2.5 sometime this week with those changes.
>>
>> For those who care about why we support unsafe and dig into the runtime
>> internals, please read below.
>>
>> To illustrate the benefit, look at unsafe vs codec.safe benchmark results
>> below:
>> ```
>>  tags: "" (default high-performance mode using unsafe carefully) 
>> Benchmark__Json___Encode-86921 152808 ns/op   24 B/op
>>  1 allocs/op
>> Benchmark__Json___Encode-85622 197863 ns/op10048 B/op
>>381 allocs/op
>>
>>  tags: "codec.safe" 
>> Benchmark__Json___Decode-82587 415595 ns/op71878 B/op
>>592 allocs/op
>> Benchmark__Json___Decode-82167 478122 ns/op96812 B/op
>>   1456 allocs/op
>>
>>  tags: x generated 
>> Benchmark__Json___Encode-88694 120519 ns/op 1528 B/op
>>  6 allocs/op
>> Benchmark__Json___Decode-82960 349320 ns/op70469 B/op
>>589 allocs/op
>> ```
>>
>> This benchmarks show that using unsafe carefully can cut down allocations
>> dramatically. Encoding allocation goes from 381 to 1, while decoding goes
>> from 1456 to 592. Those numbers using unsafe rival the allocation numbers
>> that we get using code-generation (as seen above), and the run time starts
>> to trend within 25% of the code-generation numbers, and 25% better than the
>> non-unsafe run time.
>>
>> We limited the surface area that is exposed to unsafe (basically 1 file,
>> helper_unsafe.go, with some variant for gccgo/gollvm where some linkname
>> references do not exist or work differently), so we can quickly edit the
>> code and know where bugs are coming from. Many other packages that try to
>> optimize json/cbor/msgpack encoding and decoding use some variation of the
>> same ideas here.
>>
>> I test the code  for the last 5 go releases on each github checkin, via
>> travis CI. I also run with the standard compiler and gccgo locally before
>> cutting a release. I have now added gollvm to my pre-release validation
>> script, so it is validated before I cut the release. Caveat: I test with
>> the installed versions of gccgo from ubuntu, and locally built gollvm.
>> Since gollvm is not released yet, the version I test with may be old
>> (building gollvm takes roughly 1 hour on my computer).
>>
>> If you see any further issues, please file a bug and I will jump on it:
>> https://github.com/ugorji/go/issues/new
>>
>> Thanks.
>> On Monday, March 29, 2021 at 8:32:53 AM UTC-4 peterGo wrote:
>>
>>> You haven't said whether you followed the "safe" instructions for
>>> github.com/ugorji/go/codec to avoid building code/helper_unsafe.go,
>>> which uses go:linkname.
>>>
>>> Package Documentation for github.com/ugorji/go/codec
>>> https://github.com/ugorji/go/blob/master/codec/README.md
>>>
>>> This package will carefully use 'package unsafe' for performance reasons
>>> in specific places. You can build without unsafe use by passing the safe or
>>> appengine tag i.e. 'go install -tags=codec.safe
>>>
>>> You can run the tag 'codec.safe' to run tests or build in safe mode. e.g.
>>>
>>> go test -tags codec.safe -run Json
>>> go test -tags "alltests codec.safe" -run Suite
>>>
>>> Peter
>>>
>>> On Monday, March 29, 2021 at 3:34:44 AM UTC-4 f011...@gmail.com wrote:
>>>
 >  go:linkname to refer directly to functions that are defined but not
 exported by the standard library.
 > This is not supported and is likely to break with any new release. It
 evidently breaks with GoLLVM.

 Thanks for your attention, but I tried to write a demo with go:linkname.
 In fact, it works well with gollvm...So maybe it is not the exact cause
 for the problem

 Herei is my code:

 hello/hello.go
 

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

['Than McIntosh' via golang-nuts]

Thank you @Ugorji Nwoke   , I appreciate your going the
extra mile to include gollvm in your testing.

Cheers, Than


On Tue, Mar 30, 2021 at 11:17 AM Ugorji Nwoke  wrote:

> Ugorji here - author of the github.com/ugorji/go/codec
>  package.
>
> The notes below are for folks that are interested in why we use unsafe,
> and how we mitigate concerns around it.
>
> As Peter mentioned above, you can pass the build tag "codec.safe" to
> bypass using unsafe code where we try to reach deeper into the runtime
> implementation to optimize.
>
> I have been supporting gccgo for a while now, but never tested with
> gollvm, as it has not been released at this time. I built a gollvm version
> yesterday (from source) and tested it, and made some changes (see here
> 
> and here
> )
> so that gollvm will work in the default (high-performance) mode. I plan to
> cut a new release v1.2.5 sometime this week with those changes.
>
> For those who care about why we support unsafe and dig into the runtime
> internals, please read below.
>
> To illustrate the benefit, look at unsafe vs codec.safe benchmark results
> below:
> ```
>  tags: "" (default high-performance mode using unsafe carefully) 
> Benchmark__Json___Encode-86921 152808 ns/op   24 B/op
>  1 allocs/op
> Benchmark__Json___Encode-85622 197863 ns/op10048 B/op
>381 allocs/op
>
>  tags: "codec.safe" 
> Benchmark__Json___Decode-82587 415595 ns/op71878 B/op
>592 allocs/op
> Benchmark__Json___Decode-82167 478122 ns/op96812 B/op
>   1456 allocs/op
>
>  tags: x generated 
> Benchmark__Json___Encode-88694 120519 ns/op 1528 B/op
>  6 allocs/op
> Benchmark__Json___Decode-82960 349320 ns/op70469 B/op
>589 allocs/op
> ```
>
> This benchmarks show that using unsafe carefully can cut down allocations
> dramatically. Encoding allocation goes from 381 to 1, while decoding goes
> from 1456 to 592. Those numbers using unsafe rival the allocation numbers
> that we get using code-generation (as seen above), and the run time starts
> to trend within 25% of the code-generation numbers, and 25% better than the
> non-unsafe run time.
>
> We limited the surface area that is exposed to unsafe (basically 1 file,
> helper_unsafe.go, with some variant for gccgo/gollvm where some linkname
> references do not exist or work differently), so we can quickly edit the
> code and know where bugs are coming from. Many other packages that try to
> optimize json/cbor/msgpack encoding and decoding use some variation of the
> same ideas here.
>
> I test the code  for the last 5 go releases on each github checkin, via
> travis CI. I also run with the standard compiler and gccgo locally before
> cutting a release. I have now added gollvm to my pre-release validation
> script, so it is validated before I cut the release. Caveat: I test with
> the installed versions of gccgo from ubuntu, and locally built gollvm.
> Since gollvm is not released yet, the version I test with may be old
> (building gollvm takes roughly 1 hour on my computer).
>
> If you see any further issues, please file a bug and I will jump on it:
> https://github.com/ugorji/go/issues/new
>
> Thanks.
> On Monday, March 29, 2021 at 8:32:53 AM UTC-4 peterGo wrote:
>
>> You haven't said whether you followed the "safe" instructions for
>> github.com/ugorji/go/codec to avoid building code/helper_unsafe.go,
>> which uses go:linkname.
>>
>> Package Documentation for github.com/ugorji/go/codec
>> https://github.com/ugorji/go/blob/master/codec/README.md
>>
>> This package will carefully use 'package unsafe' for performance reasons
>> in specific places. You can build without unsafe use by passing the safe or
>> appengine tag i.e. 'go install -tags=codec.safe
>>
>> You can run the tag 'codec.safe' to run tests or build in safe mode. e.g.
>>
>> go test -tags codec.safe -run Json
>> go test -tags "alltests codec.safe" -run Suite
>>
>> Peter
>>
>> On Monday, March 29, 2021 at 3:34:44 AM UTC-4 f011...@gmail.com wrote:
>>
>>> >  go:linkname to refer directly to functions that are defined but not
>>> exported by the standard library.
>>> > This is not supported and is likely to break with any new release. It
>>> evidently breaks with GoLLVM.
>>>
>>> Thanks for your attention, but I tried to write a demo with go:linkname.
>>> In fact, it works well with gollvm...So maybe it is not the exact cause
>>> for the problem
>>>
>>> Herei is my code:
>>>
>>> hello/hello.go
>>> ```
>>> package hello
>>>
>>> import (
>>> "fmt"
>>> _ "unsafe"
>>> )
>>> //go:linkname hellox hello.hellox
>>> func hellox(x string) {
>>> fmt.Println(x)
>>> }
>>> ```
>>>
>>> main.go
>>> ```
>>> package main
>>>
>>> import (
>>> _ 

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

[Ugorji Nwoke]

Ugorji here - author of the github.com/ugorji/go/codec 
 package.

The notes below are for folks that are interested in why we use unsafe, and 
how we mitigate concerns around it.

As Peter mentioned above, you can pass the build tag "codec.safe" to bypass 
using unsafe code where we try to reach deeper into the runtime 
implementation to optimize.

I have been supporting gccgo for a while now, but never tested with gollvm, 
as it has not been released at this time. I built a gollvm version 
yesterday (from source) and tested it, and made some changes (see here 
 
and here 
) 
so that gollvm will work in the default (high-performance) mode. I plan to 
cut a new release v1.2.5 sometime this week with those changes.

For those who care about why we support unsafe and dig into the runtime 
internals, please read below.

To illustrate the benefit, look at unsafe vs codec.safe benchmark results 
below:
```
 tags: "" (default high-performance mode using unsafe carefully) 
Benchmark__Json___Encode-86921 152808 ns/op   24 B/op   
 1 allocs/op
Benchmark__Json___Encode-85622 197863 ns/op10048 B/op   
   381 allocs/op

 tags: "codec.safe" 
Benchmark__Json___Decode-82587 415595 ns/op71878 B/op   
   592 allocs/op
Benchmark__Json___Decode-82167 478122 ns/op96812 B/op   
  1456 allocs/op

 tags: x generated 
Benchmark__Json___Encode-88694 120519 ns/op 1528 B/op   
 6 allocs/op
Benchmark__Json___Decode-82960 349320 ns/op70469 B/op   
   589 allocs/op
```

This benchmarks show that using unsafe carefully can cut down allocations 
dramatically. Encoding allocation goes from 381 to 1, while decoding goes 
from 1456 to 592. Those numbers using unsafe rival the allocation numbers 
that we get using code-generation (as seen above), and the run time starts 
to trend within 25% of the code-generation numbers, and 25% better than the 
non-unsafe run time.

We limited the surface area that is exposed to unsafe (basically 1 file, 
helper_unsafe.go, with some variant for gccgo/gollvm where some linkname 
references do not exist or work differently), so we can quickly edit the 
code and know where bugs are coming from. Many other packages that try to 
optimize json/cbor/msgpack encoding and decoding use some variation of the 
same ideas here.

I test the code  for the last 5 go releases on each github checkin, via 
travis CI. I also run with the standard compiler and gccgo locally before 
cutting a release. I have now added gollvm to my pre-release validation 
script, so it is validated before I cut the release. Caveat: I test with 
the installed versions of gccgo from ubuntu, and locally built gollvm. 
Since gollvm is not released yet, the version I test with may be old 
(building gollvm takes roughly 1 hour on my computer).

If you see any further issues, please file a bug and I will jump on 
it: https://github.com/ugorji/go/issues/new

Thanks.
On Monday, March 29, 2021 at 8:32:53 AM UTC-4 peterGo wrote:

> You haven't said whether you followed the "safe" instructions for 
> github.com/ugorji/go/codec to avoid building code/helper_unsafe.go, which 
> uses go:linkname.
>
> Package Documentation for github.com/ugorji/go/codec
> https://github.com/ugorji/go/blob/master/codec/README.md
>
> This package will carefully use 'package unsafe' for performance reasons 
> in specific places. You can build without unsafe use by passing the safe or 
> appengine tag i.e. 'go install -tags=codec.safe 
>
> You can run the tag 'codec.safe' to run tests or build in safe mode. e.g.
>
> go test -tags codec.safe -run Json
> go test -tags "alltests codec.safe" -run Suite
>
> Peter
>
> On Monday, March 29, 2021 at 3:34:44 AM UTC-4 f011...@gmail.com wrote:
>
>> >  go:linkname to refer directly to functions that are defined but not 
>> exported by the standard library. 
>> > This is not supported and is likely to break with any new release. It 
>> evidently breaks with GoLLVM.
>>
>> Thanks for your attention, but I tried to write a demo with go:linkname.
>> In fact, it works well with gollvm...So maybe it is not the exact cause 
>> for the problem
>>
>> Herei is my code:
>>
>> hello/hello.go
>> ```
>> package hello
>>
>> import (
>> "fmt"
>> _ "unsafe"
>> )
>> //go:linkname hellox hello.hellox
>> func hellox(x string) {
>> fmt.Println(x)
>> }
>> ```
>>
>> main.go
>> ```
>> package main
>>
>> import (
>> _ "mypackage/hello"
>> _ "unsafe"
>> )
>>
>> //go:linkname hel hello.hellox
>> func hel(x string)
>>
>> func main() {
>> hel("aaa")
>> //println("")
>> }
>> ```
>>
>> 在2021年3月29日星期一 UTC+8 上午12:37:49 写道：
>>
>>> On Sun, Mar 28, 2021 at 9:28 AM 张嘉熙  wrote: 
>>> > 
>>> > For s2-geojson, I meet: 
>>> > 

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

[JX Zhang]

> The Go runtime and standard library has a public, exported API; this sort 
of use of go:linkname to reach into the internals of a standard package (in 
this case, calling an unexported function) is not supported.
The limitation to access private functions in std package sounds reasonable.

> You can run the tag 'codec.safe' to run tests or build in safe mode. e.g.
>  go test -tags codec.safe -run Json
>  go test -tags "alltests codec.safe" -run Suite
 
thanks for your advice, but I'm not aiming at this project. I just wonder 
whether gollvm is adaptable to large-scale projects and why this case fails 
is enough
^ _ ^

The problem cause by reflect.unsafe_New can be seen solved. Welcome to add 
something if anyone has a better answer~
---

Then I wonder why gollvm work failed with pthread_join which is a C lib 
function
I wrote  a demo:

```
package main

/*
#cgo CFLAGS: -I./
#cgo LDFLAGS: -L./
#include "ccode.h"
*/
import "C"
import (
"fmt"
)

func main() {
v := C.try_pthread()
fmt.Println(v)
}
```

```
# include 
# include "ccode.h"
# include 

int try_pthread(){
pthread_t id;
int ret, i = 0;
ret = pthread_create(,NULL,(void *)thread,NULL);
for(i=0;i<=5;i++)
{
printf("This is main thread %d\n",i);
//sleep(1);
}
pthread_join(id,NULL);
//gets();
return 0;
}

void thread()
{
int i=0;
for(i=0;i<=5;i++)
{
printf("this is thread %d\n",i);
// sleep();
}
}
```

pthread_create() works well in gollvm but pthread_join() failed...
在2021年3月29日星期一 UTC+8 下午8:40:34 写道：

> On Monday, March 29, 2021 at 8:32:53 AM UTC-4 peterGo wrote:
>
>> You haven't said whether you followed the "safe" instructions for 
>> github.com/ugorji/go/codec to avoid building code/helper_unsafe.go, 
>> which uses go:linkname.
>>
>>
> s/code/helper_unsafe.go/codec/helper_unsafe.go/
>
> Peter
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/93edfaf0-2d32-4496-aaa5-9b3b60cf23b6n%40googlegroups.com.

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

[peterGo]

On Monday, March 29, 2021 at 8:32:53 AM UTC-4 peterGo wrote:

> You haven't said whether you followed the "safe" instructions for 
> github.com/ugorji/go/codec to avoid building code/helper_unsafe.go, which 
> uses go:linkname.
>
>
s/code/helper_unsafe.go/codec/helper_unsafe.go/

Peter

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/cb6ec6ca-c82e-4c9a-be47-751162ba7386n%40googlegroups.com.

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

[peterGo]

You haven't said whether you followed the "safe" instructions for 
github.com/ugorji/go/codec to avoid building code/helper_unsafe.go, which 
uses go:linkname.

Package Documentation for github.com/ugorji/go/codec
https://github.com/ugorji/go/blob/master/codec/README.md

This package will carefully use 'package unsafe' for performance reasons in 
specific places. You can build without unsafe use by passing the safe or 
appengine tag i.e. 'go install -tags=codec.safe 

You can run the tag 'codec.safe' to run tests or build in safe mode. e.g.

go test -tags codec.safe -run Json
go test -tags "alltests codec.safe" -run Suite

Peter

On Monday, March 29, 2021 at 3:34:44 AM UTC-4 f011...@gmail.com wrote:

> >  go:linkname to refer directly to functions that are defined but not 
> exported by the standard library. 
> > This is not supported and is likely to break with any new release. It 
> evidently breaks with GoLLVM.
>
> Thanks for your attention, but I tried to write a demo with go:linkname.
> In fact, it works well with gollvm...So maybe it is not the exact cause 
> for the problem
>
> Herei is my code:
>
> hello/hello.go
> ```
> package hello
>
> import (
> "fmt"
> _ "unsafe"
> )
> //go:linkname hellox hello.hellox
> func hellox(x string) {
> fmt.Println(x)
> }
> ```
>
> main.go
> ```
> package main
>
> import (
> _ "mypackage/hello"
> _ "unsafe"
> )
>
> //go:linkname hel hello.hellox
> func hel(x string)
>
> func main() {
> hel("aaa")
> //println("")
> }
> ```
>
> 在2021年3月29日星期一 UTC+8 上午12:37:49 写道：
>
>> On Sun, Mar 28, 2021 at 9:28 AM 张嘉熙  wrote: 
>> > 
>> > For s2-geojson, I meet: 
>> > ``` 
>> > # github.com/pantrif/s2-geojson/cmd/s2-geojson 
>> > 
>> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>>  
>> github_0com_1ugorji_1go_1codec.intf2impls.intf2impl: error: undefined 
>> reference to 'reflect.unsafe_New' 
>> > 
>> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>>  
>> github_0com_1ugorji_1go_1codec.Decoder.interfaceExtConvertAndDecode: error: 
>> undefined reference to 'reflect.unsafe_New' 
>> > decode.go:509: error: undefined reference to 'reflect.unsafe_New' 
>> > decode.go:14935: error: undefined reference to 'reflect.unsafe_New' 
>>
>> This is a problem with github.com/ugorji/go. The file 
>> code/helper_unsafe.go uses go:linkname to refer directly to functions 
>> that are defined but not exported by the standard library. This is 
>> not supported and is likely to break with any new release. It 
>> evidently breaks with GoLLVM. 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f12e94e5-9968-48fb-9ce6-d9899a3796e2n%40googlegroups.com.

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

['Than McIntosh' via golang-nuts]

Hi,

*>Thanks for your attention, but I tried to write a demo with go:linkname.*
*>In fact, it works well with gollvm...So maybe it is not the exact cause
for the problem*

The problem is not that gollvm fails to implement go:linkname-- the problem
is the way that this package (http://github.com/ugorji/go)  is using
go:linkname.

The Go runtime and standard library has a public, exported API; this sort
of use of go:linkname to reach into the internals of a standard package (in
this case, calling an unexported function) is not supported.

Folks who write such code should understand that it can break at any time
from release to release or across Go implementations.

Than




On Mon, Mar 29, 2021 at 3:35 AM JX Zhang  wrote:

> >  go:linkname to refer directly to functions that are defined but not
> exported by the standard library.
> > This is not supported and is likely to break with any new release. It
> evidently breaks with GoLLVM.
>
> Thanks for your attention, but I tried to write a demo with go:linkname.
> In fact, it works well with gollvm...So maybe it is not the exact cause
> for the problem
>
> Herei is my code:
>
> hello/hello.go
> ```
> package hello
>
> import (
> "fmt"
> _ "unsafe"
> )
> //go:linkname hellox hello.hellox
> func hellox(x string) {
> fmt.Println(x)
> }
> ```
>
> main.go
> ```
> package main
>
> import (
> _ "mypackage/hello"
> _ "unsafe"
> )
>
> //go:linkname hel hello.hellox
> func hel(x string)
>
> func main() {
> hel("aaa")
> //println("")
> }
> ```
>
> 在2021年3月29日星期一 UTC+8 上午12:37:49 写道：
>
>> On Sun, Mar 28, 2021 at 9:28 AM 张嘉熙  wrote:
>> >
>> > For s2-geojson, I meet:
>> > ```
>> > # github.com/pantrif/s2-geojson/cmd/s2-geojson
>> >
>> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>> github_0com_1ugorji_1go_1codec.intf2impls.intf2impl: error: undefined
>> reference to 'reflect.unsafe_New'
>> >
>> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>> github_0com_1ugorji_1go_1codec.Decoder.interfaceExtConvertAndDecode: error:
>> undefined reference to 'reflect.unsafe_New'
>> > decode.go:509: error: undefined reference to 'reflect.unsafe_New'
>> > decode.go:14935: error: undefined reference to 'reflect.unsafe_New'
>>
>> This is a problem with github.com/ugorji/go. The file
>> code/helper_unsafe.go uses go:linkname to refer directly to functions
>> that are defined but not exported by the standard library. This is
>> not supported and is likely to break with any new release. It
>> evidently breaks with GoLLVM.
>>
>> Ian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/5d763cdf-1147-4abe-aea7-9f6efa39e930n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2BUr55H4Q4%2BoztZjm3ZcrsFdQ-gBvy_V9WE%3D-Y3CnoTHWLFuqg%40mail.gmail.com.

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

[JX Zhang]

>  go:linkname to refer directly to functions that are defined but not 
exported by the standard library. 
> This is not supported and is likely to break with any new release. It 
evidently breaks with GoLLVM.

Thanks for your attention, but I tried to write a demo with go:linkname.
In fact, it works well with gollvm...So maybe it is not the exact cause for 
the problem

Herei is my code:

hello/hello.go
```
package hello

import (
"fmt"
_ "unsafe"
)
//go:linkname hellox hello.hellox
func hellox(x string) {
fmt.Println(x)
}
```

main.go
```
package main

import (
_ "mypackage/hello"
_ "unsafe"
)

//go:linkname hel hello.hellox
func hel(x string)

func main() {
hel("aaa")
//println("")
}
```

在2021年3月29日星期一 UTC+8 上午12:37:49 写道：

> On Sun, Mar 28, 2021 at 9:28 AM 张嘉熙  wrote:
> >
> > For s2-geojson, I meet:
> > ```
> > # github.com/pantrif/s2-geojson/cmd/s2-geojson
> > 
> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>  
> github_0com_1ugorji_1go_1codec.intf2impls.intf2impl: error: undefined 
> reference to 'reflect.unsafe_New'
> > 
> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>  
> github_0com_1ugorji_1go_1codec.Decoder.interfaceExtConvertAndDecode: error: 
> undefined reference to 'reflect.unsafe_New'
> > decode.go:509: error: undefined reference to 'reflect.unsafe_New'
> > decode.go:14935: error: undefined reference to 'reflect.unsafe_New'
>
> This is a problem with github.com/ugorji/go. The file
> code/helper_unsafe.go uses go:linkname to refer directly to functions
> that are defined but not exported by the standard library. This is
> not supported and is likely to break with any new release. It
> evidently breaks with GoLLVM.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5d763cdf-1147-4abe-aea7-9f6efa39e930n%40googlegroups.com.

Re: [go-nuts] gollvm hasn't a good support to reflect and pthread?

[Ian Lance Taylor]

On Sun, Mar 28, 2021 at 9:28 AM 张嘉熙  wrote:
>
> For s2-geojson, I meet:
> ```
> # github.com/pantrif/s2-geojson/cmd/s2-geojson
> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>  github_0com_1ugorji_1go_1codec.intf2impls.intf2impl: error: undefined 
> reference to 'reflect.unsafe_New'
> /home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
>  github_0com_1ugorji_1go_1codec.Decoder.interfaceExtConvertAndDecode: error: 
> undefined reference to 'reflect.unsafe_New'
> decode.go:509: error: undefined reference to 'reflect.unsafe_New'
> decode.go:14935: error: undefined reference to 'reflect.unsafe_New'

This is a problem with github.com/ugorji/go.  The file
code/helper_unsafe.go uses go:linkname to refer directly to functions
that are defined but not exported by the standard library.  This is
not supported and is likely to break with any new release.  It
evidently breaks with GoLLVM.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcU6VDXs02NQwooJY63GLC3794%2BBdX__UOaTnBj5KbhX2Q%40mail.gmail.com.

[go-nuts] gollvm hasn't a good support to reflect and pthread?

[张嘉熙]

I made a issue at github, and the details are as followed:

### What version of Go are you using (`go version`)?


$ go version
go version go1.16 gollvm LLVM 13.0.0git linux/amd64


### Does this issue reproduce with the latest release?
yes


### What operating system and processor architecture are you using (`go 
env`)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/home/jx/workspace/gowork/bin"
GOCACHE="/home/jx/.cache/go-build"
GOENV="/home/jx/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jx/workspace/gowork/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jx/workspace/gowork"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct;
GOROOT="/home/jx/.local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jx/.local/gollvm/tools"
GOVCS=""
GOVERSION="go1.16 gollvm LLVM 13.0.0git"
GCCGO="/home/jx/.local/gollvm/bin/llvm-goc"
AR="ar"
CC="/home/jx/workspace/llvm_area/llvm-project/build/bin/clang"
CXX="/home/jx/workspace/llvm_area/llvm-project/build/bin/clang++"
CGO_ENABLED="1"
GOMOD="/home/jx/workspace/batch_compile/awesome-go/picfit/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments 
-fmessage-length=0 -fdebug-prefix-map=/tmp/go-build4288114194=/tmp/go-build 
-gno-record-gcc-switches -funwind-tables"



### What did you do?


I attempted to compile mounts of go projects with gollvm, but some of them 
meet reference error to `reflect` or `pthread`
Some cases are followed:
1. https://github.com/pantrif/s2-geojson
This case meet `error: undefined reference to 'reflect.unsafe_New` while 
compiling
```
git clone https://github.com/pantrif/s2-geojson.git
cd s2-geojson/cmd/s2-geojson
go build .
```
`Go` mentioned above is just gollvm

2. https://github.com/thoas/picfit
This case meet 'undefined reference to 'pthread_once' and many other 
functions start with `pthread_`
```
git clone https://github.com/thoas/picfit.git
cd picfit/cmd/picfit
go build .
```

### What did you expect to see?
Projects are compiled successfully as the original go compiler


### What did you see instead?
For s2-geojson, I meet:
```
# github.com/pantrif/s2-geojson/cmd/s2-geojson
/home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
 
github_0com_1ugorji_1go_1codec.intf2impls.intf2impl: error: undefined 
reference to 'reflect.unsafe_New'
/home/jx/.cache/go-build/6b/6bd003c99eb0a9e7c6ea6d372307b292ec615c75c28f9b1f696896ae2fb4272b-d(_go_.o):gomodule:function
 
github_0com_1ugorji_1go_1codec.Decoder.interfaceExtConvertAndDecode: error: 
undefined reference to 'reflect.unsafe_New'
decode.go:509: error: undefined reference to 'reflect.unsafe_New'
decode.go:14935: error: undefined reference to 'reflect.unsafe_New'
```

For picfit, I meet:
build details Output
# github.com/thoas/picfit/cmd/picfit
src/libavcodec/aacdec_template.c:1143: error: undefined reference to 
'pthread_once'
src/libavcodec/h264dec.c:400: error: undefined reference to 'pthread_once'
src/libavcodec/pthread_frame.c:675: error: undefined reference to 
'pthread_join'
src/libavcodec/pthread_slice.c:116: error: undefined reference to 
'pthread_join'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::Mutex::Mutex(): error: undefined reference to 'pthread_mutexattr_init'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::Mutex::Mutex(): error: undefined reference to 
'pthread_mutexattr_settype'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::Mutex::Mutex(): error: undefined reference to 
'pthread_mutexattr_destroy'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::Mutex::trylock(): error: undefined reference to 'pthread_mutex_trylock'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::TlsAbstraction::TlsAbstraction(): error: undefined reference to 
'pthread_key_create'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::TlsAbstraction::~TlsAbstraction(): error: undefined reference to 
'pthread_key_delete'
/home/jx/workspace/batch_compile/awesome-go/picfit/vendor/github.com/discordapp/lilliput/deps/linux/lib/libopencv_core.a(system.cpp.o):system.cpp:function
 
cv::TlsAbstraction::GetData() const: error: undefined reference to