date:20221101

[go-nuts] There is a passage in book that is difficult to understand, can anyone help explain it?

2022-11-01 Thread GuanYun

Hello,

This is a passage in book :

There is a good reason Go’s mutexes are not re-entrant.

The purpose of a mutex is to ensure that certain invariants of the shared
variables are
maintained at critical points during program execution.

One of the invariants is "no goroutine is accessing the shared variables",
but there may be additional invariants specific to the data structures that
the mutex guards.

When a goroutine acquires a mutex lock, it may assume that the invariants
hold. While it holds the lock, it may update the shared variables so that
the invariants are temporarily violated.

However, when it releases the lock, it must guarantee that order has been
restored
and the invariants hold once again.

Although a re-entrant mutex would ensure that no other goroutines are
accessing the shared variables, it cannot protect the additional invariants
of those variables.

This passage is difficult for me to understand:
1. How to understand invariants "invariants"?
2. What kind of scenarios does “additional invariants” refer to?
3. What is the relationship between "shared variables" and "invariants"?
4. What does "...guarantee that order has been restored..." mean?

Thanks,
Guanyun

--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/b732271d-4bd5-47d7-bdfc-9c4c612b63e1n%40googlegroups.com.

[go-nuts] Understanding some gotchas with linking slices together via indexing

2022-11-01 Thread Brian, son of Bob

Can anyone explain these gotchas (demo )?  
I've tried reading articles on this [one , 
two 
] 
but they don't go into enough detail.


*Slight gotcha #1: arr[:N] only creates a linked slice when N < len(arr) 
and arr[N:] only creates a linked slice when N>0.*
E.g. `y` is linked here: 
y := x[:len(x) - 1]
and here:
y := x[1:]

But not here
y := x[:len(x)]
or here:
y := x[0:]

Also AFAICT, it's impossible create a link to an empty/nil slice unless you 
use pointers.  And I'm not sure if it's possible to create a linked slice 
that looks at all of the elements in the original without using a pointer.

I kind of understand where the Go designers were coming from since it is 
probably more efficient to avoid copying all the time.  Though by that 
logic, I'd still think x[0:] would create a linked slice.


*Gotcha #2: Appending to a linked slice affects the original slice but not 
the other way around*
E.g. if `y` is linked to `x` via `y :=  x[1:], then:
- appending to `y` *might* affect x (see Gotcha #3!)
- appending to `x` won't affect `y`
If there is a link, why does it not work both ways?


*Gotcha #3: Using variadic args to append to a linked slice doesn't affect 
the other one*
If `y` is linked to `x` via `y :=  x[:len(x) - 1], then
- append(y, 1) affects x
- append(y, 1, 2) doesn't affect x
I really don't get this one.


Thanks for any insights!

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7e2b82cc-f5f4-4fe4-ba73-0ff4dead66f7n%40googlegroups.com.

Re: [go-nuts] Once again: Include files in go?

2022-11-01 Thread Kurtis Rader

I think you're looking for the "//go:generate" directive. Google "go
generate" and/or read documents such as
https://ehrt74.medium.com/go-generate-89b20a27f7f9 and
https://go.dev/blog/generate.

On Tue, Nov 1, 2022 at 7:44 PM Frank Jüdes  wrote:

> Hi friends, i have found a couple of threads about how to include
> c-headers into go-programs, but nothing that i could use for my rather
> unique challenge:
>
> I have an SQL-Database in which test-cases are being defined in a couple
> of tables. Those test-cases will be exported by a PL/SQL program as *go
> source-code*. Basically the Database is writing a piece of go source-code
> in which a single variable, representing a large data-structure is being
> initialized. This variable should be part of a package, defining functions
> on that structure. For now i am just copying and pasting the generated file
> into the go-package using an editor, but that will not be practical for
> much longer, as the amount of data is increasing rapidly…
> So what i am looking for is the classic c-style include:
>
> package package_name
>
> import ( … )
>
> type …
>
> #include GeneratedDataFromDB
>
> func()…
>
> it is not something my life is depending on, i can always use Linux tools
> to *cat* some files together, but a simple #include - or similar would be
> more elegant.
> Thank you very much in advance for your help.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7c1524a5-6c45-4ca8-862e-84f0b620b18fn%40googlegroups.com
> 
> .
>


-- 
Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD8Kqn1DH0rq4RjA-87N%2BKhkqSMOqJJe%3D_55fbdpSFZOZw%40mail.gmail.com.

Re: [go-nuts] Once again: Include files in go?

2022-11-01 Thread 'Christian Stewart' via golang-nuts

Frank,

Am I missing something - can't you just put multiple go files in the same
directory?

All the go files in the directory will be combined into the same package.

This is frequently used for other codegen, like Protobuf.

Best,
Christian Stewart

On Tue, Nov 1, 2022, 7:44 PM Frank Jüdes  wrote:

> Hi friends, i have found a couple of threads about how to include
> c-headers into go-programs, but nothing that i could use for my rather
> unique challenge:
>
> I have an SQL-Database in which test-cases are being defined in a couple
> of tables. Those test-cases will be exported by a PL/SQL program as *go
> source-code*. Basically the Database is writing a piece of go source-code
> in which a single variable, representing a large data-structure is being
> initialized. This variable should be part of a package, defining functions
> on that structure. For now i am just copying and pasting the generated file
> into the go-package using an editor, but that will not be practical for
> much longer, as the amount of data is increasing rapidly…
> So what i am looking for is the classic c-style include:
>
> package package_name
>
> import ( … )
>
> type …
>
> #include GeneratedDataFromDB
>
> func()…
>
> it is not something my life is depending on, i can always use Linux tools
> to *cat* some files together, but a simple #include - or similar would be
> more elegant.
> Thank you very much in advance for your help.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7c1524a5-6c45-4ca8-862e-84f0b620b18fn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA%2Bh8R2pp4TccU1cBO-cnhPDiQQ-8EmMmfwGEVd%3DNZwXJUQ7dCg%40mail.gmail.com.

Re: [go-nuts] How does the go compiler treat initialized string variables?

2022-11-01 Thread Frank Jüdes

Just had an idea: I printed the address of the initialized variable, and 
for comparison another variable that was created in the heap:
Address of package variable: 0x818710
Address of Program variable: 0xc10028
Imho it is a safe assumption that the initialized data-structure is not 
located in the heap, but somewhere in a text-segment.

Frank Jüdes schrieb am Dienstag, 1. November 2022 um 22:15:27 UTC-4:

> Thank you very much for the answer! - It actually turns out that my 
> structure is a bit more complex than i though. The test-cases themself are 
> a structure of seven strings and one in64 which are organized as a 
> test-case list into a map[string] and those lists are organized into groups 
> also as maps[strings]… 勞
> I am generating the variable that holds all this data straight out of two 
> Database-tables, it looks like this
> var TestCaseGroup = T_TestCaseGroup {
>   "cn=config": {
> "ds-cfg-add-missing-rdn-attributes": {
>   RecommendedValue: "true",
>   MessageClass: "Recommendation",
>   MessageType: "Compatibility",
>   MessageText: "It is recommended to enable this feature to make OUD 
> more compatible to older applications."},
> "ds-cfg-allow-attribute-name-exceptions": {
>   RecommendedValue: "false",
>   MessageClass: "Severe Warning",
>   MessageType: "Data-Quality",
>   MessageText: "This feature should be disabled, to prevent the 
> directory-schema to become incompatible with LDAP standards."},
> …
> And i have no idea how to figure out if these strings are being copied 
> into the heap.
> But: The good news is, that the compiler is performing a string 
> de-duplication, for example the string "Mild Error" appears in hundred of 
> test-cases but appears only once in the whole program-code. - Tested with 
> strings | grep 'Mild Error'. I think that's a good sign.
> Jan Mercl schrieb am Dienstag, 1. November 2022 um 15:47:47 UTC-4:
>
>>
>>
>> On Tue, Nov 1, 2022, 20:36 Frank Jüdes  wrote:
>>
>>> I have to write a program that should verify the content of 
>>> configuration-servers. It will need a lot of pre-initialized data to verify 
>>> the actual content of a server, so it has to initialize many strings. What 
>>> i know from other C-style languages is, that code like 
>>>
>>> var MyString *string = 'Hello World!'; 
>>>
>>> Will result in having two identical copies of the string »Hello World!« 
>>> in the memory: The first one within the program-code itself and a second 
>>> copy somewhere in the heap-memory.
>>>
>>
>> I think the string backing array will be in the text segment as in C. The 
>> string header will end in the data segment, provided it's a package scoped 
>> variable, but the header has no direct equivalent in C.
>>
>> How will the go-compiler handle something like this: 
>>>
>>> package main 
>>>   import ("fmt") 
>>>   type MyStruct struct { 
>>> Text string 
>>> Count int32 
>>>   } 
>>>   func main() { 
>>> MyVar := MyStruct{Text: "Hello World!", Count: 20 } 
>>> fmt.Printf("%#v\n",MyVar) } 
>>>
>>> Will there be two copies of the string »Hello World!" in the memory or 
>>> just one? 
>>>
>>
>> The backing string  array will exist only once, again in the text 
>> segment, I believe, because there's no reason for making any copies of it 
>> in this case.
>>
>>>
>> Not tested/verified 
>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/535a5550-1c8c-4eaf-9d17-baa5a8d73e8bn%40googlegroups.com.

[go-nuts] Once again: Include files in go?

2022-11-01 Thread Frank Jüdes

Hi friends, i have found a couple of threads about how to include c-headers 
into go-programs, but nothing that i could use for my rather unique 
challenge:

I have an SQL-Database in which test-cases are being defined in a couple of 
tables. Those test-cases will be exported by a PL/SQL program as *go 
source-code*. Basically the Database is writing a piece of go source-code 
in which a single variable, representing a large data-structure is being 
initialized. This variable should be part of a package, defining functions 
on that structure. For now i am just copying and pasting the generated file 
into the go-package using an editor, but that will not be practical for 
much longer, as the amount of data is increasing rapidly…
So what i am looking for is the classic c-style include:
 
package package_name

import ( … )

type …

#include GeneratedDataFromDB

func()…

it is not something my life is depending on, i can always use Linux tools 
to *cat* some files together, but a simple #include - or similar would be 
more elegant.
Thank you very much in advance for your help.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7c1524a5-6c45-4ca8-862e-84f0b620b18fn%40googlegroups.com.

Re: [go-nuts] How does the go compiler treat initialized string variables?

2022-11-01 Thread Frank Jüdes

Thank you very much for the answer! - It actually turns out that my 
structure is a bit more complex than i though. The test-cases themself are 
a structure of seven strings and one in64 which are organized as a 
test-case list into a map[string] and those lists are organized into groups 
also as maps[strings]… 勞
I am generating the variable that holds all this data straight out of two 
Database-tables, it looks like this
var TestCaseGroup = T_TestCaseGroup {
  "cn=config": {
"ds-cfg-add-missing-rdn-attributes": {
  RecommendedValue: "true",
  MessageClass: "Recommendation",
  MessageType: "Compatibility",
  MessageText: "It is recommended to enable this feature to make OUD 
more compatible to older applications."},
"ds-cfg-allow-attribute-name-exceptions": {
  RecommendedValue: "false",
  MessageClass: "Severe Warning",
  MessageType: "Data-Quality",
  MessageText: "This feature should be disabled, to prevent the 
directory-schema to become incompatible with LDAP standards."},
…
And i have no idea how to figure out if these strings are being copied into 
the heap.
But: The good news is, that the compiler is performing a string 
de-duplication, for example the string "Mild Error" appears in hundred of 
test-cases but appears only once in the whole program-code. - Tested with 
strings | grep 'Mild Error'. I think that's a good sign.
Jan Mercl schrieb am Dienstag, 1. November 2022 um 15:47:47 UTC-4:

>
>
> On Tue, Nov 1, 2022, 20:36 Frank Jüdes  wrote:
>
>> I have to write a program that should verify the content of 
>> configuration-servers. It will need a lot of pre-initialized data to verify 
>> the actual content of a server, so it has to initialize many strings. What 
>> i know from other C-style languages is, that code like 
>>
>> var MyString *string = 'Hello World!'; 
>>
>> Will result in having two identical copies of the string »Hello World!« 
>> in the memory: The first one within the program-code itself and a second 
>> copy somewhere in the heap-memory.
>>
>
> I think the string backing array will be in the text segment as in C. The 
> string header will end in the data segment, provided it's a package scoped 
> variable, but the header has no direct equivalent in C.
>
> How will the go-compiler handle something like this: 
>>
>> package main 
>>   import ("fmt") 
>>   type MyStruct struct { 
>> Text string 
>> Count int32 
>>   } 
>>   func main() { 
>> MyVar := MyStruct{Text: "Hello World!", Count: 20 } 
>> fmt.Printf("%#v\n",MyVar) } 
>>
>> Will there be two copies of the string »Hello World!" in the memory or 
>> just one? 
>>
>
> The backing string  array will exist only once, again in the text segment, 
> I believe, because there's no reason for making any copies of it in this 
> case.
>
>>
> Not tested/verified 
>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/a7d42a59-aafe-4fad-8988-940301aa8db5n%40googlegroups.com.

Re: [go-nuts] Underlying type constraints and pointers

2022-11-01 Thread 'Aaron Beitch' via golang-nuts

Thanks for your help here! You helped me realize the thing I really want 
access to is the methods of Foo and Bar, and not the struct fields (my 
example was misleading).

What I would really like to have is a constraint that the type must have an 
underlying type of Foo, but that isn't working how I want. So, what I can 
do instead is define an interface that includes the methods that Foo 
provides and use that as my constraint. https://go.dev/play/p/BwEb8Lm4pb6

Aaron

On Friday, October 21, 2022 at 10:44:57 PM UTC-7 axel.wa...@googlemail.com 
wrote:

> Actually (maybe I should stop, think, and then E-Mail) this works: 
> https://go.dev/play/p/SBDglnj8mQx
> I'm not sure if the selector X(*a).data will actually copy the value, but 
> I don't *think* it does.
> I also tried this first: https://go.dev/play/p/3qUhyOkvcVm
> The fact that this doesn't work looks like a bug to me, to be honest.
>
> On Sat, Oct 22, 2022 at 7:36 AM Axel Wagner  
> wrote:
>
>> Sorry, I literally just fell out of bed, so I jumped on the first thing I 
>> saw, which probably isn't the most helpful answer.
>> You might be interested in this sentence from the Go 1.18 release notes 
>> :
>>
>> The Go compiler does not support accessing a struct field x.f where x is 
>>> of type parameter type even if all types in the type parameter's type set 
>>> have a field f. We may remove this restriction in a future release.
>>
>>  
>> So while what I said is correct - the underlying type of Bar *isnt* Foo - 
>> I think the more disappointing fact is that what you want to do just won't 
>> work for now. It might in the future, but not right now.
>>
>> On Sat, Oct 22, 2022 at 7:12 AM Axel Wagner  
>> wrote:
>>
>>> (or, for convenience, you can use a type alias `type foo = struct{ data 
>>> []string }`, which should allow you to write ~foo.
>>>
>>> On Sat, Oct 22, 2022 at 7:11 AM Axel Wagner  
>>> wrote:
>>>


 On Sat, Oct 22, 2022 at 1:40 AM 'Aaron Beitch' via golang-nuts <
 golan...@googlegroups.com> wrote:

> I have types Foo and Bar. Bar's underlying type is Foo.
>

 The underlying type is defined recursively and always either a 
 predeclared type (int, string…), or a type literal.
 The underlying type of Bar is the underlying type of Foo, which is 
 struct { data []string }. You might try using `~struct{ data []string }`.
  

> I want a function that takes in pointers to types whose underlying 
> type is Foo, but I get a compile error when trying to call this function:
>
> *Bar does not implement ~*Foo (*Bar missing in ~*main.Foo)
>
> Should this work?
>
> My real use case is around a hash map library. It's implementation is 
> a copy of Go's built-in map, but allows the user to control the equal and 
> hash function, thus allowing keys of any type rather than just comparable 
> types. github.com/aristanetworks/gomap
>
> I would like to provide helper functions 
>  for 
> gomap.Map similar to those provided by golang.org/x/exp/maps and for 
> those helpers to work on pointers to types whose underlying type is 
> gomap.Map.
>
> Thanks,
> Aaron
>
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/d6c4621a-13d8-4cd2-b777-eef7a200cca2n%40googlegroups.com
>  
> 
> .
>


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5b049c16-2dfc-43eb-bde1-e9627a08ab27n%40googlegroups.com.

Re: [go-nuts] How do I set a pkg's version?

2022-11-01 Thread Chris Burkert

During CI/CD we create a json file with a few details (git tag, branch,
hash, date, time). Afterwards we compile Go Code which embeds this file
into the binary. During runtime flags like --version print the json.

Note that this is about the version of some binary - not the version of a
package. However, you could embed go.mod. But there may be better ways.

Hope this helps.

'Mark' via golang-nuts  schrieb am Di. 1.
Nov. 2022 um 16:34:

> I am creating a pkg.
> It has a `go.mod` file:
> ```
> module github.com/.../mypkg
>
> go 1.19
> ```
> And code files with functions in .go files all in the mypkg pkg.
>
> How do I specify which version the mypkg pkg is? I know about using `$
> git tag vX.Y.Z` and `$ git push origin vX.Y.Z`, but is there any way I can
> have this version in a file that can be accessed at build time or runtime?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/0a16b738-59e5-4885-90c8-cd168e308623n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CALWqRZov4r%2BN2FiZug5mmwUwhYcvf08922UQU%3DMqfJKLFT8dBg%40mail.gmail.com.

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

2022-11-01 Thread Roland Müller

Hello,

Am Dienstag, 1. November 2022 schrieb Robert Engels :
> Also, enabling and generating core dumps is somewhat OS version specific
- even between Linux versions - so you probably need to check your docs.
>

dumping core files is a kernel feature, but I think most Linux distros have
it disabled nowadays.

To allow unlimited core files enter following command as root:
# ulimit -c unlimited

BR,
Roland

> On Nov 1, 2022, at 11:14 AM, Robert Engels  wrote:
>
> 
> Are you certain you aren’t integrating with some C library that is
interfering with the default signal handling?
>
> On Nov 1, 2022, at 11:12 AM, Steven Sokol  wrote:
>
> Yep, I need to figure out how to generate a core dump. I've tried all
the usual methods without success. Any suggestions for alternate methods
would be greatly appreciated.
> When the process is killed (SIGKILL) the system does return to its normal
load. (Under production conditions systemd immediately restarts it.)
>
> On Tuesday, November 1, 2022 at 10:38:12 AM UTC-5 ren...@ix.netcom.com
wrote:
>>
>> Perf doesn’t help. Perf only shows a sampling over time (eg where the
cpu has spent) and as you see in the logs it doesn’t work for pre-existing
threads.
>> You need to figure out how to generate a core dump.
>> When you go process dies does the system function as normal?
>>
>> On Oct 31, 2022, at 11:52 AM, Steven Sokol  wrote:
>>
>> Ok, here's what perf shows:
>>
>> root@pi4cm1(rw):/# perf record -a -g sleep 5
>> Reading /proc/4199/task/4199/maps time out. You may want to increase the
time limit by --proc-map-timeout
>> [ perf record: Woken up 8 times to write data ]
>> Warning:
>> 1 map information files for pre-existing threads were
>> not processed, if there are samples for addresses they
>> will not be resolved, you may find out which are these
>> threads by running with -v and redirecting the output
>> to a file.
>> The time limit to process proc map is too short?
>> Increase it by --proc-map-timeout
>> [ perf record: Captured and wrote 6.510 MB perf.data (80088 samples) ]
>>
>>
>>
>>┌─Warning:─┐
>>│1 map information files for pre-existing threads were │
>>│not processed, if there are samples for addresses they│
>>│will not be resolved, you may find out which are these│
>>│threads by running with -v and redirecting the output │
>>│to a file.│
>>│The time limit to process proc map is too short?  │
>>│Increase it by --proc-map-timeout │
>>│  │
>>│  │
>>│Press any key...  │
>>└──┘
>>
>>
>>
>> Samples: 80K of event 'cpu-clock:pppH', Event count (approx.):
2002200
>>   Children  Self  Command  Shared Object Symbol
>> +   41.96% 0.00%  fvUnisocket  [vectors] [.]
0x0fc0
>> +   41.65%41.65%  fvUnisocket  [vectors] [.]
0x0fc0
>> +   30.14%29.94%  fvUnisocket  fvUnisocket   [.]
kernelcas
>> +5.05% 0.00%  fvUnisocket  [vectors] [.]
0x0fa0
>> +5.04% 5.01%  fvUnisocket  fvUnisocket   [.]
runtime/internal/atomic.Cas
>> +5.01% 5.01%  fvUnisocket  [vectors] [.]
0x0fa0
>> +4.56% 4.53%  fvUnisocket  fvUnisocket   [.]
runtime/internal/atomic.goLoad64
>> +2.97% 0.00%  fvUnisocket  [vectors] [.]
0x0fd8
>> +2.95% 2.95%  fvUnisocket  [vectors] [.]
0x0fd8
>> +2.52% 0.00%  fvUnisocket  [vectors] [.]
0x0fcc
>> +2.50% 2.50%  fvUnisocket  [vectors] [.]
0x0fcc
>> +2.26% 0.00%  swapper  [kernel.kallsyms] [k]
cpu_startup_entry
>> +2.25% 0.00%  swapper  [kernel.kallsyms] [k]
do_idle
>> +2.12% 0.00%  swapper  [kernel.kallsyms] [k]
default_idle_call
>> +2.09% 2.09%  swapper  [kernel.kallsyms] [k]
arch_cpu_idle
>> +1.74% 0.00%  swapper  [unknown] [k]
0x002018ac
>> +1.74% 0.00%  swapper  [kernel.kallsyms] [k]
secondary_start_kernel
>> +1.48% 1.47%  fvUnisocket  fvUnisocket   [.] cas
>> +1.19% 1.19%  fvUnisocket  fvUnisocket   [.]
runtime/internal/atomic.goStore64
>>  0.96% 0.95%  dump1090 dump1090  [.]
convert_uc8_nodc_nopower
>> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k]
__irq_usr
>> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k]
gic_handle_irq
>> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k]
__handle_domain_irq
>> +

Re: [go-nuts] How does the go compiler treat initialized string variables?

2022-11-01 Thread Jan Mercl

On Tue, Nov 1, 2022, 20:36 Frank Jüdes  wrote:

> I have to write a program that should verify the content of
> configuration-servers. It will need a lot of pre-initialized data to verify
> the actual content of a server, so it has to initialize many strings. What
> i know from other C-style languages is, that code like
>
> var MyString *string = 'Hello World!';
>
> Will result in having two identical copies of the string »Hello World!« in
> the memory: The first one within the program-code itself and a second copy
> somewhere in the heap-memory.
>

I think the string backing array will be in the text segment as in C. The
string header will end in the data segment, provided it's a package scoped
variable, but the header has no direct equivalent in C.

How will the go-compiler handle something like this:
>
> package main
>   import ("fmt")
>   type MyStruct struct {
> Text string
> Count int32
>   }
>   func main() {
> MyVar := MyStruct{Text: "Hello World!", Count: 20 }
> fmt.Printf("%#v\n",MyVar) }
>
> Will there be two copies of the string »Hello World!" in the memory or
> just one?
>

The backing string  array will exist only once, again in the text segment,
I believe, because there's no reason for making any copies of it in this
case.

>
Not tested/verified

>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-XoZNA%2Bfu4G2X%2Bar9pJ4RyrEfYsbY2YmgvGeU%2BeUCwDTQ%40mail.gmail.com.

[go-nuts] How does the go compiler treat initialized string variables?

2022-11-01 Thread Frank Jüdes

 I have to write a program that should verify the content of 
configuration-servers. It will need a lot of pre-initialized data to verify 
the actual content of a server, so it has to initialize many strings. What 
i know from other C-style languages is, that code like 

var MyString *string = 'Hello World!'; 

Will result in having two identical copies of the string »Hello World!« in 
the memory: The first one within the program-code itself and a second copy 
somewhere in the heap-memory.
How will the go-compiler handle something like this: 

package main 
  import ("fmt") 
  type MyStruct struct { 
Text string 
Count int32 
  } 
  func main() { 
MyVar := MyStruct{Text: "Hello World!", Count: 20 } 
fmt.Printf("%#v\n",MyVar) } 

Will there be two copies of the string »Hello World!" in the memory or just 
one? As said, mMy code will contain a gazillion of pre-defined string 
variables and having each string allocate two copies of itself in memory 
would be bad for small systems. 
Thank you very much in advance for your help. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/20c9e775-fee8-4ac4-90e7-032ce37909e9n%40googlegroups.com.

[go-nuts] Re: [security] Go 1.19.3 and Go 1.18.8 are released

2022-11-01 Thread Wojciech Kaczmarek

I think it's important to report that  go1.18.8.windows-amd64.msi 
 doesn't match the published 
sha256 checksum. 


wtorek, 1 listopada 2022 o 18:01:35 UTC+1 anno...@golang.org napisał(a):

> Hello gophers,
>
> We have just released Go versions 1.19.3 and 1.18.8, minor point releases.
>
> These minor releases include 1 security fixes following the security 
> policy :
>
>- 
>
>syscall, os/exec: unsanitized NUL in environment variables
>
>On Windows, syscall.StartProcess and os/exec.Cmd did not properly 
>check for invalid environment variable values. A malicious environment 
>variable value could exploit this behavior to set a value for a different 
>environment variable. For example, the environment variable string 
>"A=B\x00C=D" set the variables "A=B" and "C=D".
>
>Thanks to RyotaK (https://twitter.com/ryotkak) for reporting this 
>issue.
>
>This is CVE-2022-41716 and Go issue https://go.dev/issue/56284.
>
> View the release notes for more information:
> https://go.dev/doc/devel/release#go1.19.3
>
> You can download binary and source distributions from the Go website:
> https://go.dev/dl/
>
> To compile from source using a Git clone, update to the release with
> git checkout go1.19.3 and build as usual.
>
> Thanks to everyone who contributed to the releases.
>
> Cheers,
> Matthew and Heschi for the Go team
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b3a092a1-38a2-452b-9451-ceda7ab0cdcdn%40googlegroups.com.

[go-nuts] Go Crashes on a long running process (with stack trace)

2022-11-01 Thread Vineet Jain

We have a long running process that listens to and processes a large number 
of network packets. It died today with the following stack trace. Anyone 
seen this before, or have any idea what is causing it?

Nov 01 12:40:30  sv_svc[100448]: I1101 12:40:30.865268  100448 
stratconn.go:83] Message Sequence(109845000): Q Length(1)
Nov 01 12:40:31  sv_svc[100448]: n on lineno rlimitpanicwrap: unexpected 
string after ty
Nov 01 12:40:31  sv_svc[100448]: 
erfreecr1742248535156257105427357601001nTpUpd0x1NFOTyp0x0 pc=0x0]
Nov 01 12:40:31  sv_svc[100448]: goroutine 4 [wner di]:
Nov 01 12:40:31  sv_svc[100448]: runtime.throw({0x75b1b0?, 0x0?})
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/panic.go:1047 +0x5d 
fp=0xcc8080 sp=0xcc8050 pc=0x436a1d
Nov 01 12:40:31  sv_svc[100448]: runtime: g 4invalid use of 
scannermalforuntime.sigpanic_ERRORGC scav0x0
Nov 01 12:40:31  sv_svc[100448]: olding lockssync.0xcc8080, 
fp:0xcc80d0} stack=[0xcc7800,0xcc8000)
Nov 01 12:40:31  sv_svc[100448]: 0x00cc7f80:  0x000b08be 
 0x412f0ff6
Nov 01 12:40:31  sv_svc[100448]: 0x00cc7f90:  0x00cc7fb8 
 0x0042449f 
Nov 01 12:40:31  sv_svc[100448]: 0x00cc7fa0:  0x00a071c0 
 0x412f0ff6
Nov 01 12:40:31  sv_svc[100448]: 0x00cc7fb0:  0x00cf 
 0x00cc7fd0
Nov 01 12:40:31  sv_svc[100448]: 0x00cc7fc0:  0x0041ac46 
  0x
Nov 01 12:40:31  sv_svc[100448]: 0x00cc7fd0:  0x 
 0x00467501 
Nov 01 12:40:31  sv_svc[100448]: 0x00cc7fe0:  0x 
 0x
Nov 01 12:40:31  sv_svc[100448]: 0x00cc7ff0:  0x 
 0x
Nov 01 12:40:31  sv_svc[100448]: runtime.sigpanic()
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/signal_unix.go:819 +0x369 
fp=0xcc80d0 sp=0xcc8080 pc=0x44cee9
Nov 01 12:40:31  sv_svc[100448]: created by runtime.gcenable
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/mgc.go:179 +0xaa
Nov 01 12:40:31  sv_svc[100448]: goroutine 1 [semacquire, 1562.locktra]:
Nov 01 12:40:31  sv_svc[100448]: runtime.gopark(0xc0027ee400?, 
0x7f8110208f50?, 0x80?, 0x76?, 0x7f81103445b8?)
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/proc.go:363 +0xd6 
fp=0xc000810d20 sp=0xc000810d00 pc=0x439636
Nov 01 12:40:31  sv_svc[100448]: runtime.goparkunlockcopyst   
 /go/src/runtime/proc.go:369
Nov 01 12:40:31  sv_svc[100448]: runtime.semacquire1(0xa37928, 0xd0?, 0x1, 
0x0)
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/sema.go:150 +0x1fe 
fp=0xc000810d88 sp=0xc000810d20 pc=0x44a31e
Nov 01 12:40:31  sv_svc[100448]: sync.runtime_Semacquire(0xc0005222c0?)
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/sema.go:62 +0x25 
fp=0xc000810db8 sp=0xc000810d88 pc=0x4635e5
Nov 01 12:40:31  sv_svc[100448]: sync.(*WaitGroup).Wait(0x77359400?)
Nov 01 12:40:31  sv_svc[100448]: /go/src/sync/waitgroup.go:139 +0x52 
fp=0xc000810de0 sp=0xc000810db8 pc=0x46f4f2
Nov 01 12:40:31  sv_svc[100448]: main.main()
Nov 01 12:40:31  sv_svc[100448]: /sv/svgo/feed/cmd/mcast/main.go:72 +0x753 
fp=0xc000810f80 sp=0xc000810de0 pc=0x6b5293
Nov 01 12:40:31  sv_svc[100448]: runtime.main()
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/proc.go:250 +0x212 
fp=0xc000810fe0 sp=0xc000810f80 pc=0x439272
Nov 01 12:40:31  sv_svc[100448]: runtime.goexit()
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/asm_amd64.s:1594 +0x1 
fp=0xc000810fe8 sp=0xc000810fe0 pc=0x467501
Nov 01 12:40:31  sv_svc[100448]: goroutine 2 [force gc (idle), 2.locktra]:
Nov 01 12:40:31  sv_svc[100448]: runtime.gopark(0x31b5261ac10cf2?, 0x0?, 
0x0?, 0x0?, 0x0?)
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/proc.go:363 +0xd6 
fp=0xcc6fb0 sp=0xcc6f90 pc=0x439636
Nov 01 12:40:31  sv_svc[100448]: runtime.goparkunlockcopyst   
 /go/src/runtime/proc.go:369
Nov 01 12:40:31  sv_svc[100448]: runtime.forcegchelper()
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/proc.go:302 +0xad 
fp=0xcc6fe0 sp=0xcc6fb0 pc=0x4394cd
Nov 01 12:40:31  sv_svc[100448]: runtime.goexit()
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/asm_amd64.s:1594 +0x1 
fp=0xcc6fe8 sp=0xcc6fe0 pc=0x467501
Nov 01 12:40:31  sv_svc[100448]: created by runtime.init.6
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/proc.go:290 +0x25
Nov 01 12:40:31  sv_svc[100448]: goroutine 3 [ionbad specia]:
Nov 01 12:40:31  sv_svc[100448]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 
0x0?)
Nov 01 12:40:31  sv_svc[100448]: /go/src/runtime/proc.go:363 +0xd6 
fp=0xcc7790 sp=0xcc7770 pc=0x439636
Nov 01 12:40:31  sv_svc[100448]: n on lineno r font file formatb
Nov 01 12:40:31  sv_svc[100448]: pflag %q begins wit
Nov 01 12:40:31  sv_svc[100448]: runtime stack:

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit

[go-nuts] [security] Go 1.19.3 and Go 1.18.8 are released

2022-11-01 Thread announce

Hello gophers,

We have just released Go versions 1.19.3 and 1.18.8, minor point releases.

These minor releases include 1 security fixes following the security policy 
:

-   syscall, os/exec: unsanitized NUL in environment variables

On Windows, syscall.StartProcess and os/exec.Cmd did not properly check 
for invalid environment variable values. A malicious environment variable value 
could exploit this behavior to set a value for a different environment 
variable. For example, the environment variable string "A=B\x00C=D" set the 
variables "A=B" and "C=D".

Thanks to RyotaK (https://twitter.com/ryotkak) for reporting this issue.

This is CVE-2022-41716 and Go issue https://go.dev/issue/56284.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.19.3

You can download binary and source distributions from the Go website:
https://go.dev/dl/

To compile from source using a Git clone, update to the release with
git checkout go1.19.3 and build as usual.

Thanks to everyone who contributed to the releases.

Cheers,
Matthew and Heschi for the Go team

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7y8pwj12Q-29HThhsZFP_g%40geopod-ismtpd-6-2.

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

2022-11-01 Thread Robert Engels

Also, enabling and generating core dumps is somewhat OS version specific - even 
between Linux versions - so you probably need to check your docs. 

> On Nov 1, 2022, at 11:14 AM, Robert Engels  wrote:
> 
> 
> Are you certain you aren’t integrating with some C library that is 
> interfering with the default signal handling?
> 
>>> On Nov 1, 2022, at 11:12 AM, Steven Sokol  wrote:
>>> 
>> Yep, I need to figure out how to generate a core dump. I've tried all the 
>> usual methods without success. Any suggestions for alternate methods would 
>> be greatly appreciated.
>> 
>> When the process is killed (SIGKILL) the system does return to its normal 
>> load. (Under production conditions systemd immediately restarts it.)
>> 
>>> On Tuesday, November 1, 2022 at 10:38:12 AM UTC-5 ren...@ix.netcom.com 
>>> wrote:
>>> Perf doesn’t help. Perf only shows a sampling over time (eg where the cpu 
>>> has spent) and as you see in the logs it doesn’t work for pre-existing 
>>> threads. 
>>> 
>>> You need to figure out how to generate a core dump. 
>>> 
>>> When you go process dies does the system function as normal?
>>> 
> On Oct 31, 2022, at 11:52 AM, Steven Sokol  wrote:
> 
 Ok, here's what perf shows:
>>> 

 root@pi4cm1(rw):/# perf record -a -g sleep 5
 Reading /proc/4199/task/4199/maps time out. You may want to increase the 
 time limit by --proc-map-timeout
 [ perf record: Woken up 8 times to write data ]
 Warning:
 1 map information files for pre-existing threads were
 not processed, if there are samples for addresses they
 will not be resolved, you may find out which are these
 threads by running with -v and redirecting the output
 to a file.
 The time limit to process proc map is too short?
 Increase it by --proc-map-timeout
 [ perf record: Captured and wrote 6.510 MB perf.data (80088 samples) ]

┌─Warning:─┐
│1 map information files for pre-existing threads were │
│not processed, if there are samples for addresses they│
│will not be resolved, you may find out which are these│
│threads by running with -v and redirecting the output │
│to a file.│
│The time limit to process proc map is too short?  │
│Increase it by --proc-map-timeout │
│  │
│  │
│Press any key...  │
└──┘

 Samples: 80K of event 'cpu-clock:pppH', Event count (approx.): 2002200
   Children  Self  Command  Shared Object Symbol
 +   41.96% 0.00%  fvUnisocket  [vectors] [.] 
 0x0fc0
 +   41.65%41.65%  fvUnisocket  [vectors] [.] 
 0x0fc0
 +   30.14%29.94%  fvUnisocket  fvUnisocket   [.] 
 kernelcas
 +5.05% 0.00%  fvUnisocket  [vectors] [.] 
 0x0fa0
 +5.04% 5.01%  fvUnisocket  fvUnisocket   [.] 
 runtime/internal/atomic.Cas
 +5.01% 5.01%  fvUnisocket  [vectors] [.] 
 0x0fa0
 +4.56% 4.53%  fvUnisocket  fvUnisocket   [.] 
 runtime/internal/atomic.goLoad64
 +2.97% 0.00%  fvUnisocket  [vectors] [.] 
 0x0fd8
 +2.95% 2.95%  fvUnisocket  [vectors] [.] 
 0x0fd8
 +2.52% 0.00%  fvUnisocket  [vectors] [.] 
 0x0fcc
 +2.50% 2.50%  fvUnisocket  [vectors] [.] 
 0x0fcc
 +2.26% 0.00%  swapper  [kernel.kallsyms] [k] 
 cpu_startup_entry
 +2.25% 0.00%  swapper  [kernel.kallsyms] [k] 
 do_idle
 +2.12% 0.00%  swapper  [kernel.kallsyms] [k] 
 default_idle_call
 +2.09% 2.09%  swapper  [kernel.kallsyms] [k] 
 arch_cpu_idle
 +1.74% 0.00%  swapper  [unknown] [k] 
 0x002018ac
 +1.74% 0.00%  swapper  [kernel.kallsyms] [k] 
 secondary_start_kernel
 +1.48% 1.47%  fvUnisocket  fvUnisocket   [.] cas
 +1.19% 1.19%  fvUnisocket  fvUnisocket   [.] 
 runtime/internal/atomic.goStore64
  0.96% 0.95%  dump1090 dump1090  [.] 
 convert_uc8_nodc_nopower
 +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
 __irq_usr
 +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
 gic_handle_irq
 +0.65% 0.00%

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

2022-11-01 Thread Robert Engels

Are you certain you aren’t integrating with some C library that is interfering 
with the default signal handling?

> On Nov 1, 2022, at 11:12 AM, Steven Sokol  wrote:
> 
> Yep, I need to figure out how to generate a core dump. I've tried all the 
> usual methods without success. Any suggestions for alternate methods would be 
> greatly appreciated.
> 
> When the process is killed (SIGKILL) the system does return to its normal 
> load. (Under production conditions systemd immediately restarts it.)
> 
>> On Tuesday, November 1, 2022 at 10:38:12 AM UTC-5 ren...@ix.netcom.com wrote:
>> Perf doesn’t help. Perf only shows a sampling over time (eg where the cpu 
>> has spent) and as you see in the logs it doesn’t work for pre-existing 
>> threads. 
>> 
>> You need to figure out how to generate a core dump. 
>> 
>> When you go process dies does the system function as normal?
>> 
 On Oct 31, 2022, at 11:52 AM, Steven Sokol  wrote:
 
>>> Ok, here's what perf shows:
>> 
>>> 
>>> 
>>> root@pi4cm1(rw):/# perf record -a -g sleep 5
>>> Reading /proc/4199/task/4199/maps time out. You may want to increase the 
>>> time limit by --proc-map-timeout
>>> [ perf record: Woken up 8 times to write data ]
>>> Warning:
>>> 1 map information files for pre-existing threads were
>>> not processed, if there are samples for addresses they
>>> will not be resolved, you may find out which are these
>>> threads by running with -v and redirecting the output
>>> to a file.
>>> The time limit to process proc map is too short?
>>> Increase it by --proc-map-timeout
>>> [ perf record: Captured and wrote 6.510 MB perf.data (80088 samples) ]
>>> 
>>> 
>>> 
>>>┌─Warning:─┐
>>>│1 map information files for pre-existing threads were │
>>>│not processed, if there are samples for addresses they│
>>>│will not be resolved, you may find out which are these│
>>>│threads by running with -v and redirecting the output │
>>>│to a file.│
>>>│The time limit to process proc map is too short?  │
>>>│Increase it by --proc-map-timeout │
>>>│  │
>>>│  │
>>>│Press any key...  │
>>>└──┘
>>> 
>>> 
>>> 
>>> Samples: 80K of event 'cpu-clock:pppH', Event count (approx.): 2002200
>>>   Children  Self  Command  Shared Object Symbol
>>> +   41.96% 0.00%  fvUnisocket  [vectors] [.] 
>>> 0x0fc0
>>> +   41.65%41.65%  fvUnisocket  [vectors] [.] 
>>> 0x0fc0
>>> +   30.14%29.94%  fvUnisocket  fvUnisocket   [.] 
>>> kernelcas
>>> +5.05% 0.00%  fvUnisocket  [vectors] [.] 
>>> 0x0fa0
>>> +5.04% 5.01%  fvUnisocket  fvUnisocket   [.] 
>>> runtime/internal/atomic.Cas
>>> +5.01% 5.01%  fvUnisocket  [vectors] [.] 
>>> 0x0fa0
>>> +4.56% 4.53%  fvUnisocket  fvUnisocket   [.] 
>>> runtime/internal/atomic.goLoad64
>>> +2.97% 0.00%  fvUnisocket  [vectors] [.] 
>>> 0x0fd8
>>> +2.95% 2.95%  fvUnisocket  [vectors] [.] 
>>> 0x0fd8
>>> +2.52% 0.00%  fvUnisocket  [vectors] [.] 
>>> 0x0fcc
>>> +2.50% 2.50%  fvUnisocket  [vectors] [.] 
>>> 0x0fcc
>>> +2.26% 0.00%  swapper  [kernel.kallsyms] [k] 
>>> cpu_startup_entry
>>> +2.25% 0.00%  swapper  [kernel.kallsyms] [k] do_idle
>>> +2.12% 0.00%  swapper  [kernel.kallsyms] [k] 
>>> default_idle_call
>>> +2.09% 2.09%  swapper  [kernel.kallsyms] [k] 
>>> arch_cpu_idle
>>> +1.74% 0.00%  swapper  [unknown] [k] 
>>> 0x002018ac
>>> +1.74% 0.00%  swapper  [kernel.kallsyms] [k] 
>>> secondary_start_kernel
>>> +1.48% 1.47%  fvUnisocket  fvUnisocket   [.] cas
>>> +1.19% 1.19%  fvUnisocket  fvUnisocket   [.] 
>>> runtime/internal/atomic.goStore64
>>>  0.96% 0.95%  dump1090 dump1090  [.] 
>>> convert_uc8_nodc_nopower
>>> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
>>> __irq_usr
>>> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
>>> gic_handle_irq
>>> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
>>> __handle_domain_irq
>>> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
>>> irq_exit
>>>  0.65% 0.17%  fvUnisocket  [kernel.kallsyms] [k] 
>>> __softirqentry_text_start
>>> +0.52% 0.00%  swapper  [kernel.kallsyms] [k]

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

2022-11-01 Thread Steven Sokol

Yep, I need to figure out how to generate a core dump. I've tried all the 
usual methods without success. Any suggestions for alternate methods would 
be greatly appreciated.

When the process is killed (SIGKILL) the system does return to its normal 
load. (Under production conditions systemd immediately restarts it.)

On Tuesday, November 1, 2022 at 10:38:12 AM UTC-5 ren...@ix.netcom.com 
wrote:

> Perf doesn’t help. Perf only shows a sampling over time (eg where the cpu 
> has spent) and as you see in the logs it doesn’t work for pre-existing 
> threads. 
>
> You need to figure out how to generate a core dump. 
>
> When you go process dies does the system function as normal?
>
> On Oct 31, 2022, at 11:52 AM, Steven Sokol  wrote:
>
> Ok, here's what perf shows:
>
>
>
> root@pi4cm1(rw):/# perf record -a -g sleep 5
> Reading /proc/4199/task/4199/maps time out. You may want to increase the 
> time limit by --proc-map-timeout
> [ perf record: Woken up 8 times to write data ]
> Warning:
> 1 map information files for pre-existing threads were
> not processed, if there are samples for addresses they
> will not be resolved, you may find out which are these
> threads by running with -v and redirecting the output
> to a file.
> The time limit to process proc map is too short?
> Increase it by --proc-map-timeout
> [ perf record: Captured and wrote 6.510 MB perf.data (80088 samples) ]
>
>
>
>┌─Warning:─┐
>│1 map information files for pre-existing threads were │
>│not processed, if there are samples for addresses they│
>│will not be resolved, you may find out which are these│
>│threads by running with -v and redirecting the output │
>│to a file.│
>│The time limit to process proc map is too short?  │
>│Increase it by --proc-map-timeout │
>│  │
>│  │
>│Press any key...  │
>└──┘
>
>
>
> Samples: 80K of event 'cpu-clock:pppH', Event count (approx.): 2002200
>   Children  Self  Command  Shared Object Symbol
> +   41.96% 0.00%  fvUnisocket  [vectors] [.] 
> 0x0fc0
> +   41.65%41.65%  fvUnisocket  [vectors] [.] 
> 0x0fc0
> +   30.14%29.94%  fvUnisocket  fvUnisocket   [.] 
> kernelcas
> +5.05% 0.00%  fvUnisocket  [vectors] [.] 
> 0x0fa0
> +5.04% 5.01%  fvUnisocket  fvUnisocket   [.] 
> runtime/internal/atomic.Cas
> +5.01% 5.01%  fvUnisocket  [vectors] [.] 
> 0x0fa0
> +4.56% 4.53%  fvUnisocket  fvUnisocket   [.] 
> runtime/internal/atomic.goLoad64
> +2.97% 0.00%  fvUnisocket  [vectors] [.] 
> 0x0fd8
> +2.95% 2.95%  fvUnisocket  [vectors] [.] 
> 0x0fd8
> +2.52% 0.00%  fvUnisocket  [vectors] [.] 
> 0x0fcc
> +2.50% 2.50%  fvUnisocket  [vectors] [.] 
> 0x0fcc
> +2.26% 0.00%  swapper  [kernel.kallsyms] [k] 
> cpu_startup_entry
> +2.25% 0.00%  swapper  [kernel.kallsyms] [k] 
> do_idle
> +2.12% 0.00%  swapper  [kernel.kallsyms] [k] 
> default_idle_call
> +2.09% 2.09%  swapper  [kernel.kallsyms] [k] 
> arch_cpu_idle
> +1.74% 0.00%  swapper  [unknown] [k] 
> 0x002018ac
> +1.74% 0.00%  swapper  [kernel.kallsyms] [k] 
> secondary_start_kernel
> +1.48% 1.47%  fvUnisocket  fvUnisocket   [.] cas
> +1.19% 1.19%  fvUnisocket  fvUnisocket   [.] 
> runtime/internal/atomic.goStore64
>  0.96% 0.95%  dump1090 dump1090  [.] 
> convert_uc8_nodc_nopower
> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
> __irq_usr
> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
> gic_handle_irq
> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
> __handle_domain_irq
> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
> irq_exit
>  0.65% 0.17%  fvUnisocket  [kernel.kallsyms] [k] 
> __softirqentry_text_start
> +0.52% 0.00%  swapper  [kernel.kallsyms] [k] 
> start_kernel
> +0.52% 0.00%  swapper  [kernel.kallsyms] [k] 
> arch_call_rest_init
> +0.52% 0.00%  swapper  [kernel.kallsyms] [k] 
> __noinstr_text_end
>  0.38% 0.02%  fvUnisocket  [kernel.kallsyms] [k] 
> tasklet_action_common.constprop.4
>  0.38% 0.00%  fvUnisocket  [kernel.kallsyms] [k]

Re: [go-nuts] Looking for some Go library or application as blueprint for concurrency stress tests

2022-11-01 Thread Robert Engels

github.com/robaho/go-concurrency-test may be of some interest. 

> On Nov 1, 2022, at 10:33 AM, Haddock  wrote:
> 
> 
> Hello,
> 
> I'm developing a little concurrency library in some language that has support 
> for fibers. That concurrency library is modeled after concurrency in Go 
> (because CSP in Go makes concurrency a lot easier). 
> 
> My own more or less elaborated test cases are running fine now, but I'm 
> missing some real heavy stress tests that would put efficiency to the test as 
> well as being able to lay open hidden bugs in process synchronization where 
> I'm not always completely sure that my implementation is 100% correct.
> 
> Therefore I'm looking for some well known and well tested tools, libraries or 
> applications written in Go making heavy use of Goroutines and channels from 
> which I could extract the concurrent operations going on as a blueprint for 
> building the stress tests for my own little concurrency library.
> 
> Any hints greatly appreciated.
> Regards, Haddock
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/dd80227b-3022-4ac1-84ed-32c7128428abn%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/AAA8E3F8-081F-4585-A53F-84ABBAF39F85%40ix.netcom.com.

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

2022-11-01 Thread Robert Engels

Perf doesn’t help. Perf only shows a sampling over time (eg where the cpu has 
spent) and as you see in the logs it doesn’t work for pre-existing threads. 

You need to figure out how to generate a core dump. 

When you go process dies does the system function as normal?

> On Oct 31, 2022, at 11:52 AM, Steven Sokol  wrote:
> 
> Ok, here's what perf shows:
> 
> root@pi4cm1(rw):/# perf record -a -g sleep 5
> Reading /proc/4199/task/4199/maps time out. You may want to increase the time 
> limit by --proc-map-timeout
> [ perf record: Woken up 8 times to write data ]
> Warning:
> 1 map information files for pre-existing threads were
> not processed, if there are samples for addresses they
> will not be resolved, you may find out which are these
> threads by running with -v and redirecting the output
> to a file.
> The time limit to process proc map is too short?
> Increase it by --proc-map-timeout
> [ perf record: Captured and wrote 6.510 MB perf.data (80088 samples) ]
> 
> 
> 
>┌─Warning:─┐
>│1 map information files for pre-existing threads were │
>│not processed, if there are samples for addresses they│
>│will not be resolved, you may find out which are these│
>│threads by running with -v and redirecting the output │
>│to a file.│
>│The time limit to process proc map is too short?  │
>│Increase it by --proc-map-timeout │
>│  │
>│  │
>│Press any key...  │
>└──┘
> 
> 
> 
> Samples: 80K of event 'cpu-clock:pppH', Event count (approx.): 2002200
>   Children  Self  Command  Shared Object Symbol
> +   41.96% 0.00%  fvUnisocket  [vectors] [.] 
> 0x0fc0
> +   41.65%41.65%  fvUnisocket  [vectors] [.] 
> 0x0fc0
> +   30.14%29.94%  fvUnisocket  fvUnisocket   [.] kernelcas
> +5.05% 0.00%  fvUnisocket  [vectors] [.] 
> 0x0fa0
> +5.04% 5.01%  fvUnisocket  fvUnisocket   [.] 
> runtime/internal/atomic.Cas
> +5.01% 5.01%  fvUnisocket  [vectors] [.] 
> 0x0fa0
> +4.56% 4.53%  fvUnisocket  fvUnisocket   [.] 
> runtime/internal/atomic.goLoad64
> +2.97% 0.00%  fvUnisocket  [vectors] [.] 
> 0x0fd8
> +2.95% 2.95%  fvUnisocket  [vectors] [.] 
> 0x0fd8
> +2.52% 0.00%  fvUnisocket  [vectors] [.] 
> 0x0fcc
> +2.50% 2.50%  fvUnisocket  [vectors] [.] 
> 0x0fcc
> +2.26% 0.00%  swapper  [kernel.kallsyms] [k] 
> cpu_startup_entry
> +2.25% 0.00%  swapper  [kernel.kallsyms] [k] do_idle
> +2.12% 0.00%  swapper  [kernel.kallsyms] [k] 
> default_idle_call
> +2.09% 2.09%  swapper  [kernel.kallsyms] [k] 
> arch_cpu_idle
> +1.74% 0.00%  swapper  [unknown] [k] 
> 0x002018ac
> +1.74% 0.00%  swapper  [kernel.kallsyms] [k] 
> secondary_start_kernel
> +1.48% 1.47%  fvUnisocket  fvUnisocket   [.] cas
> +1.19% 1.19%  fvUnisocket  fvUnisocket   [.] 
> runtime/internal/atomic.goStore64
>  0.96% 0.95%  dump1090 dump1090  [.] 
> convert_uc8_nodc_nopower
> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] __irq_usr
> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
> gic_handle_irq
> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
> __handle_domain_irq
> +0.65% 0.00%  fvUnisocket  [kernel.kallsyms] [k] irq_exit
>  0.65% 0.17%  fvUnisocket  [kernel.kallsyms] [k] 
> __softirqentry_text_start
> +0.52% 0.00%  swapper  [kernel.kallsyms] [k] 
> start_kernel
> +0.52% 0.00%  swapper  [kernel.kallsyms] [k] 
> arch_call_rest_init
> +0.52% 0.00%  swapper  [kernel.kallsyms] [k] 
> __noinstr_text_end
>  0.38% 0.02%  fvUnisocket  [kernel.kallsyms] [k] 
> tasklet_action_common.constprop.4
>  0.38% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
> tasklet_hi_action
>  0.36% 0.05%  fvUnisocket  [kernel.kallsyms] [k] 
> usb_giveback_urb_bh
>  0.31% 0.00%  fvUnisocket  [kernel.kallsyms] [k] 
> __usb_hcd_giveback_urb
>  0.29% 0.00%  fvUnisocket  [uvcvideo][k] 
> uvc_video_complete
>  0.26% 0.00%  swapper  [ipv6].rodata.str1.4  [k] .LC0
>  0.24% 0.00%  swapper

[go-nuts] How do I set a pkg's version?

2022-11-01 Thread 'Mark' via golang-nuts

I am creating a pkg.
It has a `go.mod` file:
```
module github.com/.../mypkg

go 1.19
```
And code files with functions in .go files all in the mypkg pkg.

How do I specify which version the mypkg pkg is? I know about using `$ git 
tag vX.Y.Z` and `$ git push origin vX.Y.Z`, but is there any way I can have 
this version in a file that can be accessed at build time or runtime?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0a16b738-59e5-4885-90c8-cd168e308623n%40googlegroups.com.

[go-nuts] Looking for some Go library or application as blueprint for concurrency stress tests

2022-11-01 Thread Haddock

Hello,

I'm developing a little concurrency library in some language that has 
support for fibers. That concurrency library is modeled after concurrency 
in Go (because CSP in Go makes concurrency a lot easier). 

My own more or less elaborated test cases are running fine now, but I'm 
missing some real heavy stress tests that would put efficiency to the test 
as well as being able to lay open hidden bugs in process synchronization 
where I'm not always completely sure that my implementation is 100% correct.

Therefore I'm looking for some well known and well tested tools, libraries 
or applications written in Go making heavy use of Goroutines and channels 
from which I could extract the concurrent operations going on as a 
blueprint for building the stress tests for my own little concurrency 
library.

Any hints greatly appreciated.
Regards, Haddock

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/dd80227b-3022-4ac1-84ed-32c7128428abn%40googlegroups.com.

Re: [go-nuts] GoLLVM: using the result of `go build -x . ` to rebuild the program got failed

2022-11-01 Thread 'Qingwei Li' via golang-nuts

  Ok, I solved my problem again. The solution is to add `-S -emit-llvm` to 
get llvm IR for each package and use `llvm-link -S a.ll b.ll -o main.ll` to 
link all llvm IR files to get a whole program llvm IR. 

the command is as follows:

```
WORK=/tmp/go-build438096694
mkdir -p $WORK/b002/
cd $WORK
/data/mygo/gollvm_workarea/install/bin/llvm-goc -fgo-importcfg=/dev/null -c 
-x c - -o /dev/null || true
/data/mygo/gollvm_workarea/install/bin/llvm-goc -ffile-prefix-map=a=b -c -x 
c - -o /dev/null || true
cd /home/lqw/easyPkg/Pkg
/data/mygo/gollvm_workarea/install/bin/llvm-goc -c -O2 -g -m64 
-fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches 
-fgo-pkgpath=easyPkg/Pkg -o $WORK/b002/_go_.o -I $WORK/b002/_importcfgroot_ 
./utils.go
/data/mygo/gollvm_workarea/install/bin/llvm-goc -c -S -emit-llvm -O2 -g 
-m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches 
-fgo-pkgpath=easyPkg/Pkg -o $WORK/b002/_go_.ll -I 
$WORK/b002/_importcfgroot_ ./utils.go
echo '  .section .go.buildid,"e"' >> $WORK/b002/_buildid.s
echo '  .byte 0x58,0x63,0x75,0x74,0x74,0x43,0x71,0x46' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x36,0x2d,0x77,0x48,0x67,0x50,0x64,0x36' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x77,0x57,0x44,0x45,0x2f,0x58,0x63,0x75' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x74,0x74,0x43,0x71,0x46,0x36,0x2d,0x77' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x48,0x67,0x50,0x64,0x36,0x77,0x57,0x44' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x45' >> $WORK/b002/_buildid.s
echo '  .section .note.GNU-stack,"",@progbits' >> $WORK/b002/_buildid.s
echo '  .section .note.GNU-split-stack,"",@progbits' >> 
$WORK/b002/_buildid.s
echo '' >> $WORK/b002/_buildid.s
/data/mygo/gollvm_workarea/install/bin/llvm-goc -xassembler-with-cpp -I 
$WORK/b002/ -c -o $WORK/b002/_buildid.o -D GOOS_linux -D GOARCH_amd64 -D 
GOPKGPATH=easyPkg_1Pkg -m64 $WORK/b002/_buildid.s
ar rcD $WORK/b002/_pkg_.a $WORK/b002/_go_.o $WORK/b002/_buildid.o
/data/mygo/gollvm_workarea/install/tools/buildid -w $WORK/b002/_pkg_.a # 
internal
cp $WORK/b002/_pkg_.a 
/home/lqw/.cache/go-build/99/99c687f035658ebacf01fe0034d140a1c9e79ce207f0efd169fcbe0d4f23634d-d
 
# internal
mkdir -p $WORK/b001/
cat >$WORK/b001/_gomod_.go << 'EOF' # internal
package main
import _ "unsafe"
//go:linkname __set_debug_modinfo__ runtime.setmodinfo
func __set_debug_modinfo__(string)
func init() { 
__set_debug_modinfo__("0w\xaf\f\x92t\b\x02A\xe1\xc1\a\xe6\xd6\x18\xe6path\teasyPkg\nmod\teasyPkg\t(devel)\t\nbuild\t-compiler=gccgo\nbuild\tCGO_ENABLED=1\nbuild\tCGO_CFLAGS=\nbuild\tCGO_CPPFLAGS=\nbuild\tCGO_CXXFLAGS=\nbuild\tCGO_LDFLAGS=\nbuild\tGOARCH=amd64\nbuild\tGOEXPERIMENT=fieldtrack\nbuild\tGOOS=linux\nbuild\tGOAMD64=v1\n\xf92C1\x86\x18
 
r\x00\x82B\x10A\x16\xd8\xf2") }
EOF
mkdir -p $WORK/b001/_importcfgroot_/easyPkg
ln -s $WORK/b002/_pkg_.a $WORK/b001/_importcfgroot_/easyPkg/libPkg.a
cd /home/lqw/easyPkg
/data/mygo/gollvm_workarea/install/bin/llvm-goc -c -O2 -g -m64 
-fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o 
$WORK/b001/_go_.o -I $WORK/b001/_importcfgroot_ ./main.go 
$WORK/b001/_gomod_.go
/data/mygo/gollvm_workarea/install/bin/llvm-goc -c -S -emit-llvm -O2 -g 
-m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o 
$WORK/b001/_go_.ll -I $WORK/b001/_importcfgroot_ ./main.go 
$WORK/b001/_gomod_.go
llvm-link -S $WORK/b002/_go_.ll $WORK/b001/_go_.ll -o main.ll
echo '  .section .go.buildid,"e"' >> $WORK/b001/_buildid.s
echo '  .byte 0x62,0x6c,0x7a,0x44,0x45,0x34,0x43,0x38' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x56,0x42,0x6a,0x4e,0x6a,0x74,0x4e,0x68' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x66,0x71,0x4b,0x56,0x2f,0x62,0x6c,0x7a' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x44,0x45,0x34,0x43,0x38,0x56,0x42,0x6a' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x4e,0x6a,0x74,0x4e,0x68,0x66,0x71,0x4b' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x56' >> $WORK/b001/_buildid.s
echo '  .section .note.GNU-stack,"",@progbits' >> $WORK/b001/_buildid.s
echo '  .section .note.GNU-split-stack,"",@progbits' >> 
$WORK/b001/_buildid.s
echo '' >> $WORK/b001/_buildid.s
/data/mygo/gollvm_workarea/install/bin/llvm-goc -xassembler-with-cpp -I 
$WORK/b001/ -c -o $WORK/b001/_buildid.o -D GOOS_linux -D GOARCH_amd64 -m64 
$WORK/b001/_buildid.s
ar rcD $WORK/b001/_pkg_.a $WORK/b001/_go_.o $WORK/b001/_buildid.o
/data/mygo/gollvm_workarea/install/tools/buildid -w $WORK/b001/_pkg_.a # 
internal
cp $WORK/b001/_pkg_.a 
/home/lqw/.cache/go-build/db/dbe11e7450e893503846799700cb9d0005a545c180087da01751b0ba8586d06d-d
 
# internal
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile easyPkg=$WORK/b001/_pkg_.a
packagefile easyPkg/Pkg=$WORK/b002/_pkg_.a
modinfo 
"0w\xaf\f\x92t\b\x02A\xe1\xc1\a\xe6\xd6\x18\xe6path\teasyPkg\nmod\teasyPkg\t(devel)\t\nbuild\t-compiler=gccgo\nbuild\tCGO_ENABLED=1\nbuild\tCGO_CFLAGS=\nbuild\tCGO_CPPFLAGS=\nbuild\tCGO_CXXFLAGS=\nbuild\tCGO_LDFLAGS=\nbuild\tGOARCH=amd64\nbuild\tGOEXPERIMENT=fieldtrack\nbuild\tGOOS=linux\nbuild\tGOAMD64=v1\n\xf92C1\x86\x18

Re: [go-nuts] GoLLVM: using the result of `go build -x . ` to rebuild the program got failed

2022-11-01 Thread 'Qingwei Li' via golang-nuts

Ok, I solved my problem again. The solution is to add `-S -emit-llvm` to 
get llvm IR for each package and use `llvm-link -S a.ll b.ll -o main.ll` to 
link all llvm IR files to get a whole program llvm IR. 

the command is as follows:

```bash
WORK=/tmp/go-build2352156760
mkdir -p $WORK/b002/
cd $WORK
/data/mygo/gollvm_workarea/install/bin/llvm-goc -fgo-importcfg=/dev/null -c 
-x c - -o /dev/null || true
/data/mygo/gollvm_workarea/install/bin/llvm-goc -ffile-prefix-map=a=b -c -x 
c - -o /dev/null || true
cd /home/lqw/pureGo/tool
/data/mygo/gollvm_workarea/install/bin/llvm-goc -c -emit-llvm -O2 -g -m64 
-fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches 
-fgo-pkgpath=pureGo/tool -o $WORK/b002/_go_.o -I $WORK/b002/_importcfgroot_ 
./tool.go
echo '.section .go.buildid,"e"' >> $WORK/b002/_buildid.s
echo '.byte 0x4e,0x6b,0x67,0x49,0x35,0x37,0x74,0x53' >> 
$WORK/b002/_buildid.s
echo '.byte 0x72,0x39,0x4b,0x69,0x45,0x47,0x77,0x6b' >> 
$WORK/b002/_buildid.s
echo '.byte 0x38,0x74,0x6e,0x63,0x2f,0x4e,0x6b,0x67' >> 
$WORK/b002/_buildid.s
echo '.byte 0x49,0x35,0x37,0x74,0x53,0x72,0x39,0x4b' >> 
$WORK/b002/_buildid.s
echo '.byte 0x69,0x45,0x47,0x77,0x6b,0x38,0x74,0x6e' >> 
$WORK/b002/_buildid.s
echo '.byte 0x63' >> $WORK/b002/_buildid.s
echo '.section .note.GNU-stack,"",@progbits' >> $WORK/b002/_buildid.s
echo '.section .note.GNU-split-stack,"",@progbits' >> 
$WORK/b002/_buildid.s
echo '' >> $WORK/b002/_buildid.s
/data/mygo/gollvm_workarea/install/bin/llvm-goc -xassembler-with-cpp -I 
$WORK/b002/ -c -o $WORK/b002/_buildid.o -D GOOS_linux -D GOARCH_amd64 -D 
GOPKGPATH=pureGo_1tool -m64 $WORK/b002/_buildid.s
ar rcD $WORK/b002/_pkg_.a $WORK/b002/_go_.o $WORK/b002/_buildid.o
/data/mygo/gollvm_workarea/install/tools/buildid -w $WORK/b002/_pkg_.a # 
internal
cp $WORK/b002/_pkg_.a 
/home/lqw/.cache/go-build/41/41b41a918a4b1686b742b8ee1c96ea1760145f7396a1c6ecb4bd8b08434e6cf6-d
 
# internal
mkdir -p $WORK/b001/
cat >$WORK/b001/_gomod_.go << 'EOF' # internal
package main
import _ "unsafe"
//go:linkname __set_debug_modinfo__ runtime.setmodinfo
func __set_debug_modinfo__(string)
func init() { 
__set_debug_modinfo__("0w\xaf\f\x92t\b\x02A\xe1\xc1\a\xe6\xd6\x18\xe6path\tpureGo\nmod\tpureGo\t(devel)\t\nbuild\t-compiler=gccgo\nbuild\tCGO_ENABLED=1\nbuild\tCGO_CFLAGS=\nbuild\tCGO_CPPFLAGS=\nbuild\tCGO_CXXFLAGS=\nbuild\tCGO_LDFLAGS=\nbuild\tGOARCH=amd64\nbuild\tGOEXPERIMENT=fieldtrack\nbuild\tGOOS=linux\nbuild\tGOAMD64=v1\n\xf92C1\x86\x18
 
r\x00\x82B\x10A\x16\xd8\xf2") }
EOF
mkdir -p $WORK/b001/_importcfgroot_/pureGo
ln -s $WORK/b002/_pkg_.a $WORK/b001/_importcfgroot_/pureGo/libtool.a
cd /home/lqw/pureGo
/data/mygo/gollvm_workarea/install/bin/llvm-goc -c -emit-llvm -O2 -g -m64 
-fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o 
$WORK/b001/_go_.o -I $WORK/b001/_importcfgroot_ ./main.go ./other.go 
$WORK/b001/_gomod_.go
echo '.section .go.buildid,"e"' >> $WORK/b001/_buildid.s
echo '.byte 0x65,0x52,0x67,0x67,0x77,0x33,0x79,0x4c' >> 
$WORK/b001/_buildid.s
echo '.byte 0x63,0x42,0x5f,0x79,0x4c,0x62,0x4a,0x69' >> 
$WORK/b001/_buildid.s
echo '.byte 0x47,0x43,0x61,0x70,0x2f,0x65,0x52,0x67' >> 
$WORK/b001/_buildid.s
echo '.byte 0x67,0x77,0x33,0x79,0x4c,0x63,0x42,0x5f' >> 
$WORK/b001/_buildid.s
echo '.byte 0x79,0x4c,0x62,0x4a,0x69,0x47,0x43,0x61' >> 
$WORK/b001/_buildid.s
echo '.byte 0x70' >> $WORK/b001/_buildid.s
echo '.section .note.GNU-stack,"",@progbits' >> $WORK/b001/_buildid.s
echo '.section .note.GNU-split-stack,"",@progbits' >> 
$WORK/b001/_buildid.s
echo '' >> $WORK/b001/_buildid.s
/data/mygo/gollvm_workarea/install/bin/llvm-goc -xassembler-with-cpp -I 
$WORK/b001/ -c -o $WORK/b001/_buildid.o -D GOOS_linux -D GOARCH_amd64 -m64 
$WORK/b001/_buildid.s
ar rcD $WORK/b001/_pkg_.a $WORK/b001/_go_.o $WORK/b001/_buildid.o
/data/mygo/gollvm_workarea/install/tools/buildid -w $WORK/b001/_pkg_.a # 
internal
cp $WORK/b001/_pkg_.a 
/home/lqw/.cache/go-build/3a/3ac29db13716c1de2352162e6edc90665b2f4a92d63fda52799e5b7a48068d00-d
 
# internal
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile pureGo=$WORK/b001/_pkg_.a
packagefile pureGo/tool=$WORK/b002/_pkg_.a
modinfo 
"0w\xaf\f\x92t\b\x02A\xe1\xc1\a\xe6\xd6\x18\xe6path\tpureGo\nmod\tpureGo\t(devel)\t\nbuild\t-compiler=gccgo\nbuild\tCGO_ENABLED=1\nbuild\tCGO_CFLAGS=\nbuild\tCGO_CPPFLAGS=\nbuild\tCGO_CXXFLAGS=\nbuild\tCGO_LDFLAGS=\nbuild\tGOARCH=amd64\nbuild\tGOEXPERIMENT=fieldtrack\nbuild\tGOOS=linux\nbuild\tGOAMD64=v1\n\xf92C1\x86\x18
 
r\x00\x82B\x10A\x16\xd8\xf2"
EOF
mkdir -p $WORK/b001/exe/
cd .
/data/mygo/gollvm_workarea/install/bin/llvm-goc -o $WORK/b001/exe/a.out 
"-Wl,-(" -m64 -Wl,--whole-archive $WORK/b001/_pkg_.a $WORK/b002/_pkg_.a 
-Wl,--no-whole-archive "-Wl,-)" 
-Wl,--build-id=0x67495a447a3269595533515a674951655f5a59792f655267677733794c63425f794c624a69474361702f594a5044562d62346a334a42626f5842746932652f67495a447a3269595533515a674951655f5a5979

Re: [go-nuts] GoLLVM: using the result of `go build -x . ` to rebuild the program got failed

2022-11-01 Thread 'Qingwei Li' via golang-nuts

sorry, I get into trouble again. The shell script shown in this 
[post](https://groups.google.com/g/golang-nuts/c/pTX6ZtV0Uq4/m/iDz7u78_CAAJ)  
is actually what the `go build -x . 2> auto.sh` generate. But what I want 
is adding `-emit-llvm` when using `llvm-goc -c` to build source files.

the shell script I need is 

```bash
WORK=/tmp/go-build3031634242
mkdir -p $WORK/b002/
cd $WORK
/data/mygo/gollvm_workarea/install/bin/llvm-goc -fgo-importcfg=/dev/null -c 
-x c - -o /dev/null || true
/data/mygo/gollvm_workarea/install/bin/llvm-goc -ffile-prefix-map=a=b -c -x 
c - -o /dev/null || true
cd /home/lqw/easyPkg/Pkg
/data/mygo/gollvm_workarea/install/bin/llvm-goc -c -emit-llvm -O2 -g -m64 
-fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches 
-fgo-pkgpath=easyPkg/Pkg -o $WORK/b002/_go_.o -I $WORK/b002/_importcfgroot_ 
./utils.go
echo '  .section .go.buildid,"e"' >> $WORK/b002/_buildid.s
echo '  .byte 0x58,0x63,0x75,0x74,0x74,0x43,0x71,0x46' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x36,0x2d,0x77,0x48,0x67,0x50,0x64,0x36' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x77,0x57,0x44,0x45,0x2f,0x58,0x63,0x75' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x74,0x74,0x43,0x71,0x46,0x36,0x2d,0x77' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x48,0x67,0x50,0x64,0x36,0x77,0x57,0x44' >> 
$WORK/b002/_buildid.s
echo '  .byte 0x45' >> $WORK/b002/_buildid.s
echo '  .section .note.GNU-stack,"",@progbits' >> $WORK/b002/_buildid.s
echo '  .section .note.GNU-split-stack,"",@progbits' >> 
$WORK/b002/_buildid.s
echo '' >> $WORK/b002/_buildid.s
/data/mygo/gollvm_workarea/install/bin/llvm-goc -xassembler-with-cpp -I 
$WORK/b002/ -c -o $WORK/b002/_buildid.o -D GOOS_linux -D GOARCH_amd64 -D 
GOPKGPATH=easyPkg_1Pkg -m64 $WORK/b002/_buildid.s
ar rcD $WORK/b002/_pkg_.a $WORK/b002/_go_.o $WORK/b002/_buildid.o
/data/mygo/gollvm_workarea/install/tools/buildid -w $WORK/b002/_pkg_.a # 
internal
cp $WORK/b002/_pkg_.a 
/home/lqw/.cache/go-build/99/99c687f035658ebacf01fe0034d140a1c9e79ce207f0efd169fcbe0d4f23634d-d
 
# internal
mkdir -p $WORK/b001/
cat >$WORK/b001/_gomod_.go << 'EOF' # internal
package main
import _ "unsafe"
//go:linkname __set_debug_modinfo__ runtime.setmodinfo
func __set_debug_modinfo__(string)
func init() { 
__set_debug_modinfo__("0w\xaf\f\x92t\b\x02A\xe1\xc1\a\xe6\xd6\x18\xe6path\teasyPkg\nmod\teasyPkg\t(devel)\t\nbuild\t-compiler=gccgo\nbuild\tCGO_ENABLED=1\nbuild\tCGO_CFLAGS=\nbuild\tCGO_CPPFLAGS=\nbuild\tCGO_CXXFLAGS=\nbuild\tCGO_LDFLAGS=\nbuild\tGOARCH=amd64\nbuild\tGOEXPERIMENT=fieldtrack\nbuild\tGOOS=linux\nbuild\tGOAMD64=v1\n\xf92C1\x86\x18
 
r\x00\x82B\x10A\x16\xd8\xf2") }
EOF
mkdir -p $WORK/b001/_importcfgroot_/easyPkg
ln -s $WORK/b002/_pkg_.a $WORK/b001/_importcfgroot_/easyPkg/libPkg.a
cd /home/lqw/easyPkg
/data/mygo/gollvm_workarea/install/bin/llvm-goc -c -emit-llvm -O2 -g -m64 
-fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o 
$WORK/b001/_go_.o -I $WORK/b001/_importcfgroot_ ./main.go 
$WORK/b001/_gomod_.go
llvm-link -S $WORK/b002/_go_.o $WORK/b001/_go_.o -o 
/home/lqw/easyPkg/main.ll
echo '  .section .go.buildid,"e"' >> $WORK/b001/_buildid.s
echo '  .byte 0x62,0x6c,0x7a,0x44,0x45,0x34,0x43,0x38' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x56,0x42,0x6a,0x4e,0x6a,0x74,0x4e,0x68' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x66,0x71,0x4b,0x56,0x2f,0x62,0x6c,0x7a' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x44,0x45,0x34,0x43,0x38,0x56,0x42,0x6a' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x4e,0x6a,0x74,0x4e,0x68,0x66,0x71,0x4b' >> 
$WORK/b001/_buildid.s
echo '  .byte 0x56' >> $WORK/b001/_buildid.s
echo '  .section .note.GNU-stack,"",@progbits' >> $WORK/b001/_buildid.s
echo '  .section .note.GNU-split-stack,"",@progbits' >> 
$WORK/b001/_buildid.s
echo '' >> $WORK/b001/_buildid.s
/data/mygo/gollvm_workarea/install/bin/llvm-goc -xassembler-with-cpp -I 
$WORK/b001/ -c -o $WORK/b001/_buildid.o -D GOOS_linux -D GOARCH_amd64 -m64 
$WORK/b001/_buildid.s
ar rcD $WORK/b001/_pkg_.a $WORK/b001/_go_.o $WORK/b001/_buildid.o
/data/mygo/gollvm_workarea/install/tools/buildid -w $WORK/b001/_pkg_.a # 
internal
cp $WORK/b001/_pkg_.a 
/home/lqw/.cache/go-build/db/dbe11e7450e893503846799700cb9d0005a545c180087da01751b0ba8586d06d-d
 
# internal
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile easyPkg=$WORK/b001/_pkg_.a
packagefile easyPkg/Pkg=$WORK/b002/_pkg_.a
modinfo 
"0w\xaf\f\x92t\b\x02A\xe1\xc1\a\xe6\xd6\x18\xe6path\teasyPkg\nmod\teasyPkg\t(devel)\t\nbuild\t-compiler=gccgo\nbuild\tCGO_ENABLED=1\nbuild\tCGO_CFLAGS=\nbuild\tCGO_CPPFLAGS=\nbuild\tCGO_CXXFLAGS=\nbuild\tCGO_LDFLAGS=\nbuild\tGOARCH=amd64\nbuild\tGOEXPERIMENT=fieldtrack\nbuild\tGOOS=linux\nbuild\tGOAMD64=v1\n\xf92C1\x86\x18
 
r\x00\x82B\x10A\x16\xd8\xf2"
EOF
mkdir -p $WORK/b001/exe/
cd .
/data/mygo/gollvm_workarea/install/bin/llvm-goc -o $WORK/b001/exe/a.out 
"-Wl,-(" -m64 -Wl,--whole-archive $WORK/b001/_pkg_.a $WORK/b002/_pkg_.a 
-Wl,--no-whole-archive "-Wl,-)"

Re: [go-nuts] Re: Replace reference path

2022-11-01 Thread mr....@gmail.com

I think this tool just needs to be replaced from what to what.

For example this looks like this

goreplacepkg -from 
k8s.io/apimachinery/pkg/util/diff.{ObjectDiff,ObjectGoPrintDiff,ObjectGoPrintDiff}
 
-to  github.com/google/go-cmp/cmp.Diff

goreplacepkg -from ioutil.ReadAll -to  io.ReadAll

Here we think his parameters are the same, then the tool just needs to 
parse the go file, find the path of the import in this go file, and the 
methods under this path. Then do the replacement,Finally a new file is 
generated, github.com/google/go-cmp/cmp is added to import, and the 
original k8s.io/apimachinery/pkg/util/diff can be removed if he is no 
longer used.

For example, the input file would look like this

package test

import "k8s.io/apimachinery/pkg/util/diff"

func A()string{
var a = "test"
 var b = "test1"
 return diff.ObjectDiff(a,b)
}

func B()string{
  var a = "test"
  var b= 'test1"
  return diff.ObjectGoPrintSideBySide(a,b)
}



Execution of commands
goreplacepkg -from k8s.io/apimachinery/pkg/util/diff.ObjectDiff -to 
 github.com/google/go-cmp/cmp.Diff

The final file he generates may look like this

package test

import  (
"k8s.io/apimachinery/pkg/util/diff"
"github.com/google/go-cmp/cmp"
)

func A()string{
var a = "test"
 var b = "test1"
 return cmp.Diff(a,b)
}

func B()string{
  var a = "test"
  var b= 'test1"
  return diff.ObjectGoPrintSideBySide(a,b)
}


He doesn't need to know the go version, he just needs the user to pass in 
the input and output



在2022年11月1日星期二 UTC+8 13:51:07 写道：

> https://k8s.io/apimachinery/pkg/util/diff?go-get=1
>
> He needs to add parameters to be able to access
>
>
> https://github.com/kubernetes/apimachinery/blob/master/pkg/util/diff/diff.go#L57
>
> This is his path to realization
> 在2022年11月1日星期二 UTC+8 12:41:22 写道：
>
>> On Mon, Oct 31, 2022 at 8:27 PM mr@gmail.com  
>> wrote:
>>
>>> Yes, it is very rare, but it can be encountered, and once encountered 
>>> his workload will be a lot of
>>>
>>> github.com/pkg/errors > std.errors
>>> ioutil.TempDir => os.MkdirTemp
>>> ioutil.ReadAll  => io.ReadAll
>>>
>>
>> You seem to agree with me that such refactorings are extremely rare. 
>> Furthermore, your ioutil.TempDir to os.MkdirTemp example is a trivial 
>> substitution since the API didn't change -- only the preferred function 
>> name changed. How many programs include more than a single call to the now 
>> deprecated ioutil.TempDir function? And how much work is it for the people 
>> maintaining those projects to manually make the required substitutions? How 
>> much work would be saved even if those people were aware of such a tool? 
>>
>> Your real question seems to be "Is there a tool that replaces a 
>> deprecated API with the preferred API?" I am not aware of such a tool. 
>> Furthermore, such a tool would have to be aware of the minimum Go version 
>> supported by a project and whether a particular dependency can be updated. 
>> The mechanical substitution of the API is trivial compared to determining 
>> whether the substitution is appropriate. Also, the substitution is only 
>> possible if the API is unchanged -- only the preferred symbols are changed.
>>
>> I also note that none of the links in your initial message are valid. So 
>> I can't determine whether they are a trivial renaming of an API or the 
>> changes are more substantive.
>>
>>
>> 在2022年10月31日星期一 UTC+8 10:52:45 写道：
>>>
 I'm curious how often you perform such refactoring? In my experience 
 such changes are extremely rare and usually require other changes due to 
 differences in the API of the two third-party packages . Which means, in 
 my 
 experience, expending effort to automate such import rewrites typically 
 requires more effort than just doing it "by hand". The gomvpkg tool is 
 meant to solve a more common problem that does happen with some 
 regularity. 
 Namely, changing the paths of packages private to a particular project as 
 that project evolves. Which is why that tool "doesn't get the job done" 
 for 
 your situation.  

 On Sun, Oct 30, 2022 at 7:22 PM mr@gmail.com  
 wrote:

> He doesn't get the job done.
>
> 在2022年10月22日星期六 UTC+8 06:20:53 写道：
>
>> try https://pkg.go.dev/golang.org/x/tools/cmd/gomvpkg
>>
>>
>> On Friday, 21 October 2022 at 16:58:57 UTC+1 mr@gmail.com wrote:
>>
>>> I'm looking for a tool like this, I've searched google and github 
>>> and can't find one, have you guys used such a tool?
>>>
>>> I use a method like this in my code
>>>
>>> k8s.io/apimachinery/pkg/util/diff.ObjectReflectDiff
>>>
>>> k8s.io/apimachinery/pkg/util/diff is his import path, later I want 
>>> to switch to another method.
>>>
>>> github.com/google/go-cmp/cmp.Diff
>>>
>>> github.com/google/go-cmp/cmp is his import path.
>>>
>>> I would like to ask if there is a more

[go-nuts] There is a passage in book that is difficult to understand, can anyone help explain it?

[go-nuts] Understanding some gotchas with linking slices together via indexing

Re: [go-nuts] Once again: Include files in go?

Re: [go-nuts] Once again: Include files in go?

Re: [go-nuts] How does the go compiler treat initialized string variables?

[go-nuts] Once again: Include files in go?

Re: [go-nuts] How does the go compiler treat initialized string variables?

Re: [go-nuts] Underlying type constraints and pointers

Re: [go-nuts] How do I set a pkg's version?

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

Re: [go-nuts] How does the go compiler treat initialized string variables?

[go-nuts] How does the go compiler treat initialized string variables?

[go-nuts] Re: [security] Go 1.19.3 and Go 1.18.8 are released

[go-nuts] Go Crashes on a long running process (with stack trace)

[go-nuts] [security] Go 1.19.3 and Go 1.18.8 are released

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

Re: [go-nuts] Looking for some Go library or application as blueprint for concurrency stress tests

Re: [go-nuts] Occasional hard lockup w/ 100% CPU usage in go applications

[go-nuts] How do I set a pkg's version?

[go-nuts] Looking for some Go library or application as blueprint for concurrency stress tests

Re: [go-nuts] GoLLVM: using the result of `go build -x . ` to rebuild the program got failed

Re: [go-nuts] GoLLVM: using the result of `go build -x . ` to rebuild the program got failed

Re: [go-nuts] GoLLVM: using the result of `go build -x . ` to rebuild the program got failed

Re: [go-nuts] Re: Replace reference path

26 matches

Site Navigation

Mail list logo

Footer information