[go-nuts] modules (Go 1.11) and plugins

[Basile Starynkevitch]

Hello All,


https://groups.google.com/forum/#!topic/golang-nuts/swTLZyP5QK8 is a 
related but ancient thread.

*How should Go modules be used as plugins? *My intuitive feeling is that 
modules <https://github.com/golang/go/wiki/Modules> are a *dramatic* 
feature very useful for plugins. But I don't understand how to use them 
concretely (what commands should be run on Linux).

To be more specific, let's suppose I am *generating* the Go source code of 
some plugin in some fresh directory /tmp/basileplugin/ ; Assume I am able 
to generate all relevant sources files there, e.g.
/tmp/basileplugin/go.mod /tmp/basileplugin/foo.go /tmp/basileplugin/bar.go 
etc... That plugin would define a single basileplugin Go package. My main 
program don't use statically that basileplugin.
But the directory /tmp/basileplugin/ should be compiled into a 
/tmp/basileplugin.so plugin (and I don't know exactly how to do that, using 
the modules feature).
My main program is using phdl, err := plugin.Open("/tmp/basileplugin.so") 
to load that plugin. My main program define some globmain package which 
exports the globmain.Globfunc function.
My foo.go file (under /tmp/basileplugin) is defining a basileplugin.Foo 
function which calls the globmain.Globfun from my main program.
My bar.go file (under /tmp/basileplugin directory also) is defining a 
basileplugin.Bar function which calls basileplugin.Foo function.
My main program then calls phdl.Lookup("Foo") and phdl.Lookup("Bar") to get 
pointer to function (closures) and use them.

So what are the concrete commands to build the plugin /tmp/basileplugin.so 
and my main program?

Regards.

-- 
Basile Starynkevitch http://starynkevitch.net/Basile/
Bourg La Reine, France  

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] cryptic multiple roots error (in relation to plugins, -buildmode=shared, -linkshared)

[Basile Starynkevitch]

pink/pink2.og is trivially translated to its .go file pink1.go & 
   pink2.go (that file can be in whatever directory is the most 
   convenient), and all these files are defining some package pink (e.g. 
   with some public function pink.PinkFoo) and they can use standard Go 
   functions or some existing functions and packages in og.
   - a pink.so plugin is compiled from them. it is loaded by a call to 
   plugin.Open <https://tip.golang.org/pkg/plugin/>.
   - every .og file in /tmp/blue/ directory, e.g. /tmp/blue/blue1.og & 
   /tmp/blue/blue2.og is trivially translated to blue1.go & blue2.go ; 
   notice that blue2.go has a line import "pink" and is calling pink.PinkFoo 
   function from a blue.BluePublic function.
   - a blue.so plugin is compiled from them. It is loaded (by the main 
   program og). The blue.BluePublic function is looked up with plugin.Lookup 
   and executed.
   
I am able to write the trivial Og to Go compiler. What I don't understand 
is what exact commands should be run (notably what go compilation commands 
should og run).
*I'm struggling since more than a month to understand that*, so *I really 
need some help*.

Thanks for reading. Could someone explain how I can code that? (and how to 
avoid the *multiple roots* message). What would be the prefered place of 
the generated blue1.go etc...

(In monimelt, the Og source files are extracted and computed from some 
Sqlite database, but that is an implementation detail I can master). So 
both monimelt and the generated pink & blue are also using go-sqlite3 and 
other packages. 

Cheers.

Basile Starynkevitch <http://starynkevitch.net/Basile/> (France) basile at 
starynkevitch dot net




-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: package, files at build (in relation to plugins, -linkshared, -buildmode=shared, ....)

[Basile Starynkevitch]

On Saturday, April 1, 2017 at 6:38:23 PM UTC+2, Basile Starynkevitch wrote:

> Is there a way to understand where the mandatory 
> go install -buildmode=shared std
> is installing its stuff?
>
> Is there a way to check that the above command has been previously and 
> successfully run?
>


I found some way, see my monimelt/get-monimelt-dependencies.sh 
<https://github.com/bstarynk/monimelt/blob/master/get-monimelt-dependencies.sh> 
script:


mygoarch=$(go env GOARCH)  ## e.g. amd64
mygoos=$(go env GOOS)  ## e.g. linux
mygoroot=$(go env GOROOT)  ## e.g. /usr/local/go


## check that go install -buildmode=shared std has been run once
if [ -f $mygoroot/pkg/${mygoos}_${mygoarch}_dynlink/fmt.shlibname ]; then
myfmtshlibname=$(head -1 $mygoroot/pkg/${mygoos}_${mygoarch}_dynlink/fmt
.shlibname)
if [ ! -f $mygoroot/pkg/${mygoos}_${mygoarch}_dynlink/$myfmtshlibname ]; 
then
echo 1>&2
echo you should have done: go install -buildmode=shared std 1>&2
echo ... but $mygoroot/pkg/${mygoos}_${mygoarch}_dynlink/$myfmtshlibname 
is missing 1>&2
echo ... using content of $mygoroot/pkg/${mygoos}_${mygoarch}_dynlink/
$myfmtshlibname
echo >&2
exit 1
else
echo you did run: go install -buildmode=shared std
echo ... because we got: $mygoroot/pkg/${mygoos}_${mygoarch}_dynlink/fmt
.shlibname
echo ... pointing to: $mygoroot/pkg/${mygoos}_${mygoarch}_dynlink/
$myfmtshlibname
fi

else
echo 1>&2
echo you should have done: go install -buildmode=shared std 1>&2
echo ... but $mygoroot/pkg/${mygoos}_${mygoarch}_dynlink/fmt.shlibname 
is missing 1>&2
echo >&2
exit 1
fi


Now, I have to understand how to use go tool compile (probably with make). 
It looks that the go command is not suitable for me.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: package, files at build (in relation to plugins, -linkshared, -buildmode=shared, ....)

[Basile Starynkevitch]

On Saturday, April 1, 2017 at 2:59:16 AM UTC+2, Ian Lance Taylor wrote:
>
> On Fri, Mar 31, 2017 at 9:28 AM, Basile Starynkevitch 
> <bas...@starynkevitch.net > wrote: 
> > 
> > On Friday, March 31, 2017 at 4:14:19 PM UTC+2, Ian Lance Taylor wrote: 
> >> 
> >> On Fri, Mar 31, 2017 at 6:49 AM, Ian Lance Taylor <ia...@golang.org> 
> >> wrote: 
> >> > 
> >> > It depends.  The main command that generates output is `go install`, 
> >> > and it stores the output under $GOROOT/pkg. 
> >> 
> >> Whoops, I meant $GOPATH/pkg.  $GOROOT/pkg is only used for the standard 
> >> library. 
> > 
> > 
> > 
> > I'm still really confused by getting quite often a very cryptic message: 
> > 
> >  multiple roots /home/basile/go/pkg/linux_amd64 & 
> > /usr/lib/go-1.8/pkg/linux_amd64_dynlink 
> > 
> > the funny thing is that both of these are directories, not files. In my 
> > understanding that message means a conflict between two files for the 
> same 
> > package. And I have no idea about what files have been conflicting. If 
> the 
> > message is a conflict between files, why does not it name these files? 
>
> That error message means that the go tool does not know where to 
> install a file.  It may mean that you are trying to use 
> -buildmode=shared with some standard library packages and some 
> packages from your GOPATH.  It's not a conflict between different 
> files; it's talking about which directory it should put the shared 
> library in. 
>
> Ian 
>


Is there a way to understand where the mandatory 
go install -buildmode=shared std
is installing its stuff?

Is there a way to check that the above command has been previously and 
successfully run?

Is there a way to avoid the cryptic message:
multiple roots /home/basile/go/pkg/linux_amd64 &  
/usr/lib/go-1.8/pkg/linux_amd64_dynlink 

perhaps by forcing in which directory a file should be installed?

(If there is some way to avoid that multiple roots message, I would suggest 
that the error message would mention it; giving only the two words multiple 
roots as an explanation is really not enough)

Cheers.

Basile Starynkevitch
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: package, files at build (in relation to plugins, -linkshared, -buildmode=shared, ....)

[Basile Starynkevitch]

On Friday, March 31, 2017 at 6:28:40 PM UTC+2, Basile Starynkevitch wrote:

>
> I'm still really confused by getting quite often a very cryptic message:
>
>  multiple roots /home/basile/go/pkg/linux_amd64 & 
> /usr/lib/go-1.8/pkg/linux_amd64_dynlink
>
> the funny thing is that both of these are *directories*, not files. In my 
> understanding that message means a conflict between two files for the same 
> package. And I have no idea about what files have been conflicting. If the 
> message is a conflict between files, why does not it name these files?
>
>
If that helps, I have understood that I'll better use  
go install -buildmode=shared std
before any other compilation.


My point is that I really need (almost) all compilation to be 
shared-library friendly. If that was possible, I would even disable any 
kind of static (unshared) linking in my Go system


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] package, files at build (in relation to plugins, -linkshared, -buildmode=shared, ....)

[Basile Starynkevitch]

ce several weeks because of such issues, and 
I am considering (sadly) giving up Go (and sadly going back to emit C or 
C++ code at runtime, like I did in the past). I would miss Go concurrent 
abilities and its multi-thread friendly garbage collector.



My misunderstanding of Go internals -in particular with plugins and shared 
code in mind- is making me unhappy.


I am willing, if so needed, to describe in every detail what I have tried 
in my monimelt <https://github.com/bstarynk/monimelt> project (on Github).

If you are in France, in the Paris region, I can even go to your place with 
my laptop (But the people I have met at the recent Go meetup  in Paris are 
not able to help because they don't care about plugins).

Thanks for reading. Regards.

-- 
Basile Starynkevitch <http://starynkevitch.net/Basile/> (France) 
bas...@starynkevitch.net 



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: plugins (Go 1.8) and packages

[Basile Starynkevitch]

On Wednesday, March 22, 2017 at 11:00:24 PM UTC+1, Ian Lance Taylor wrote:
>
> On Wed, Mar 22, 2017 at 3:06 AM, Basile Starynkevitch 
> <bas...@starynkevitch.net > wrote: 
> > 
> > First, is Svetlin's golang-sharing-libraries tutorial still exactly 
> correct 
> > for Go1.8 specifically? Its title  Sharing Golang packages to C and Go 
> is a 
> > bit misleading. I just want all my non-main packages to be "shared 
> objects" 
> > in Linux parlance 
> > (because I really don't want them to be compiled twice, once for the 
> main 
> > program and once for the plugin). I don't care if each of my package has 
> its 
> > own *.so file, or if all of them are agglomerated in one single *.so 
> file. 
> > BTW, I would 
> > prefer to avoid having tons of *.so files (what I would probably prefer 
> is 
> > to have all the non-main code of my thing as a single *.so file shared 
> > between main program & plugins). 
>
> As far as I know that is correct. 
>
> > In particular, is issue 12236 still relevant for Go 1.8 ? 
>
> It is fixed in 1.8. 
>
> You may find it helpful to read https://golang.org/s/execmodes . 
>
>
> > I am not sure to understand well how -buildmode=shared & -linkshared 
> should 
> > be used. Is there an up to date tutorial in how to use them precisely? 
>
> Not that I know of. 
>
>
> > So how to force Go to systematically use -buildmode=shared? Is there 
> some 
> > comment directive for that? Or should I provide my own shell script to 
> build 
> > (I do know that the current one build-monimelt.sh is very buggy). 
>
> You would have to use a shell script or something.  There is no way to 
> make it the default, and really that wouldn't even make sense: you 
> would not to use -buildmode=shared when building your main executable, 
> only when building the packages that it imports. 
>
> > Is -buildmode=pie useful for me (I believe that not)? 
>
> I doubt that it is useful. 
>
> > The documentation of -linkshared  is saying just: 
> > 
> >> -linkshared 
> >> link against shared libraries previously created with 
> >> -buildmode=shared. 
> > 
> > 
> > and I find that explanation too short (what happens if by mistake a 
> previous 
> > package was compiled twice, both with -buildmode=shared & 
> -buildmode=default 
> > and what happens if two weeks ago I have compiled my purple package -in 
> some 
> > older version- with -buildmode=default and today I am compiling it -an 
> > improved version- with -buildmode=shared) ? 
>
> -linkshared is going to find the last package you built and installed 
> using `go install -buildmode=shared ...`. 
>
>

Thanks for the help. Using plugins is really difficult in practice.
 https://blog.ksub.org/bytes/2017/02/12/exploring-shared-objects-in-go/ is 
very helpful.

To Go implementors: the multiple roots error message is *really* cryptic. 
It should be a lot more explicit.

Thanks for reading

-- 
Basile Starynkevitch <http://starynkevitch.net/Basile/> (France)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: plugins (Go 1.8) and packages

[Basile Starynkevitch]

now that the current one build-monimelt.sh 
<https://github.com/bstarynk/monimelt/blob/master/build-monimelt.sh> is 
very buggy).
Is -buildmode=pie useful for me (I believe that not)?

I'm sorry for still being a newbie regarding Go. I understand that my use 
case is not usual. I have very hard time understanding the precise relation 
between Go source files and Go packages and their build procedure (notably 
for plugins).
Any link to a white paper describing them (in details) is welcome. The 
documentation of -buildmode 
<https://golang.org/cmd/go/#hdr-Description_of_build_modes> is not precise 
enough (and does not gives hints of which one to choose).   The 
documentation of -linkshared 
<https://golang.org/cmd/go/#hdr-Compile_packages_and_dependencies>  is 
saying just:

-linkshared
>   link against shared libraries previously created with
>   -buildmode=shared.
>
>
and I find that explanation too short (what happens if by mistake a 
previous package was compiled twice, both with -buildmode=shared & 
-buildmode=default and what happens if two weeks ago I have compiled my 
purple package -in some *older* version- with -buildmode=default and today 
I am compiling it -an *improved* version- with -buildmode=shared) ?

Regards

-- 
Basile Starynkevitch <http://starynkevitch.net/Basile/> (France) - email 
bas...@starynkevitch.net 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: plugins (Go 1.8) and packages

[Basile Starynkevitch]

On Monday, March 20, 2017 at 8:36:04 AM UTC+1, Basile Starynkevitch wrote:

> *Plugins and packages in Go*
>
> The *package* concept is a core concept of Go since every source file belongs 
> to some package (with main being a special case) and often imports several 
> other ones.
> Practically speaking, a Go plugin is likely to call functions from some 
> package defined by the main program and having itself (the plugin) some 
> packages. So the
>  tiny example in plugin documentation <https://tip.golang.org/pkg/plugin/> is 
> a bit too naive (even if it is calling fmt.Printf). A *realistic example* 
> would be a program having not only some main function 
> in its main package, but also defining some purple package having some Foo 
> public function called as purple.Foo from main and having a Bar public 
> function
> (with purple.Bar called from the plugin). The plugin would have not only some 
> public F function in its main package (which should call purple.Bar from the 
> main 
> plugin-loading program) but also some plugin specific yellow package with a 
> public Y function called (as yellow.Y) from F. I hope that such a realistic 
> example will be 
> given in the documentation of future Go 1.9.
>
>
>
To be more concrete, Here is the scenario I am thinking of:

the main (plugin loading) program has some purple package in a purple.go 
file (probably in some purple/ directory):
/// file purple.go of the main program
package purple
import "fmt"

func Foo(x int) { // called from main
  fmt.Printf("purple.Foo has x=%d\n", x)
}

func Bar(s string) { // called from plugin
  fmt.Printf("purple.Bar has s=%q\n", s)
}

/// eof purple.go


Let's assume we have a /tmp/plugin.so binary plugin. Is is made from 
plugintop.go and pluginyellow.go and it has a yellow package and it calls 
purple.Bar. So here is pluginyellow.go :
//file pluginyellow.go of the plugin
package yellow // the yellow package is in the plugin
import "fmt"

// we import the purple package from the main program 
import "purple"

// in real life, yellow is importing a lot more of existing packages (e.g. 
sql)

function F() { // this function is called from plugintop.go
  fmt.Printf("in yellow.F\n")
  purple.Bar("from yellow.F")
  fmt.Printf("ending yellow.F\n")
} 

function Gee () {
  fmt.Printf("in yellow.Gee calling Foo with 345\n")
  purple.Foo(345);
  fmt.Printf("end yellow.Gee")
}

function GG() {  // this ls looked up by the main program
  fmt.Printf("in yellow.GG\n")
}

// eof pluginyellow.go


And here is the top file of the plugin, plugintop.go (I don't know in what 
directory it should go exactly):
// file plugintop.go
package main

import "fmt"
import "yellow"
func init() {
  fmt.Printf("init of plugintop\n")
  yellow.F()
  fmt.Printf("end of init of plugintop\n")
}

func Pub() { // this is looked up by the main program
  fmt.Printf("in Pub of plugintop calling GG\n")
  yellow.GG()
  fmt.Printf("in Pub of plugintop ending\n")
}
// eof plugintop.go

Our main program (which is loading the plugin) defines a purple package in 
some purpleprog.go file
// file purpleprog.go
package purple
import "fmt"
func Foo(x int) {
  fmt.Printf("in purple.Foo x=%d\n", x)
}

func Bar(m string) {
  fmt.Printf("in purple.Bar m=%q\n", m)
}
// eof purpleprog.go


Of course we need a main in our program, file mainprog.go is loading the 
/tmp/plugin.so plugin
// file mainprog.go
package main
import "fmt"
import "plugin"
import "purple"

func main() {
  fmt.Printf("start of main in mainprog.go\n")
  plug, err := plugin.Open("/tmp/plugin.so")
  fmt.Printf("in mainprog plug=%v err=%v\n", plug, err)
  if err != nil {
panic(fmt.Errorf("plugin.Open /tmp/plugin.so failed in mainprog with %v"
, err))
  }
  fmt.Printf("mainprog before call purple.Foo with 12751\n")
  purple.Foo(12751)
  fmt.Printf("mainprog after call purple.Foo with 12751\n")
  symbPub, err := plug.Lookup("Pub")
  fmt.Printf("in mainprog symbPub=%v err=%v\n", symbPub, err)
  if err != nil {
panic(fmt.Errorf("Lookup of pub failed with %v", err))
  }
  funpub := symbPub.(func())
  fmt.Printf("mainprog before calling funpub=%v\n", funpub)
  funpub()
  symbGG, err := plug.Lookup("yellow.GG")
  fmt.Printf("in mainprog symbGG=%v err=%v\n", symbGG, err)
  if err != nil {
panic(fmt.Errorf("Lookup of yellow.GG failed with %v", err))
  }
  funGG := symbPub.(func())
  fmt.Printf("mainprog before calling funGG=%v\n", funGG)
  funGG()
  fmt.Printf("end of mainprog\n")
}
// end of file mainprog.go

In real life, the purple package inside t

Re: [go-nuts] []byte to func

[Basile Starynkevitch]

On Tuesday, March 21, 2017 at 4:04:34 AM UTC+1, Rob 'Commander' Pike wrote:
>
> No, Go does not have run-time evaluation of Go program source. It is 
> strictly a compiled language, although there are some (pieces of) 
> interpreters out there.
>
>
>
However, Go 1.8 has plugins <https://tip.golang.org/pkg/plugin/> (on Linux 
x86-64 at least). A possible way -which can also be used in C, C++, Ocaml, 
etc...- might be to generate (at *runtime* of your program) some Go code in 
some *temporary* /tmp/generatedsrc.go file (actually using ioutils/TempFile 
<https://golang.org/pkg/io/ioutil/#TempFile> to get it), then run (using 
functions from os/exec <https://golang.org/pkg/os/exec/> package) some 
compilation command to compile that source into a /tmp/generatedsrc.so 
shared library plugin, and Open <https://tip.golang.org/pkg/plugin/#Open> 
that plugin, then Lookup <https://tip.golang.org/pkg/plugin/#Plugin.Lookup> 
some symbol there and use a type assertion 
<https://tip.golang.org/ref/spec#Type_assertions> to convert it to some 
functional value, that could be called later.

I confess that I don't know yet exactly how to do that in a convenient and 
robust way (you want the compilation of the plugin to be minimal and quick, 
even if it uses most packages of the main program). I have started using Go 
only once it had plugins because of that issue. The issue is plugins and 
packages <https://groups.google.com/forum/#!topic/golang-nuts/IKh1BqrNoxI>. 
Another issue is what exact compilation command should be used (I suspect 
that passing -linkshared with -buildmode=plugin to the go build compilation 
command is needed).  I have asked a question about plugins and packages 
<https://groups.google.com/forum/#!topic/golang-nuts/IKh1BqrNoxI> (you 
probably don't want the plugin to incorporate all the packages of the main 
program that it is using). It would be nice if some Go plugin guru could 
explain more how to do that.

Cheers

-- 
Basile Starynkevitch <http://starynkevitch.net/Basile/> (France)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] plugins (Go 1.8) and packages

[Basile Starynkevitch]

prone* (it is not defined 
what happens if the version of that package is different, and there might 
be -in weird cases- infelicities with multiple initialization of the same 
package)




*A use case*In my monimelt <https://github.com/bstarynk/monimelt> program 
(which is now buildable with go tool and don't use gb anymore, since gb 
does not support plugins yet) commit 6bb8b56160749 
<https://github.com/bstarynk/monimelt/commit/6bb8b5616074942327748934e735e92de602f824>
the test-plugins/makename.go 
<https://github.com/bstarynk/monimelt/blob/master/test-plugins/makename.go> 
is a simplistic plugin which just calls a few functions from the objvalmo & 
payloadmo packages defined in the main program. But its compilation 
time with build-plugin-monimelt.sh 
<https://github.com/bstarynk/monimelt/blob/master/build-plugin-monimelt.sh> 
script is ridiculously big, since all the packages of the main program (and 
even external ones like go-sqlite) get linked both in the main program and 
the plugin. To run that test, do ./monimelt -run-plugin 
test-plugins/makename.go which will compile test-plugins/makename.go into 
makename.so using build-plugin-monimelt.sh script and open that plugin. The 
whole point of monimelt will be to generate Go source code (of plugins) at 
runtime and compile and load them (I followed the same idea in GCC MELT 
<http://gcc-melt.org/>).

BTW, I am interested in reading some white paper about the precise 
semantics of packages, and how the Go compiler works internally.

Cheers
-- 
Basile Starynkevitch <http://starynkevitch.net/Basile/> (France)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] watchpoint on "private" variables with Gdb

[Basile Starynkevitch]

Hello all,

I'm debugging the 5d6c770eb29ae0a33 

 
commit of my monimelt  program on 
github.
I don't understand very well how init functions work (and I probably don't 
understand the order in which init functions are running, notably with init 
functions in several files of the same package).

I'm declaring in file 

objvalmo/objvalmo.go 

 
the following package private variables:

var glovar_map map[string]**ObjectMo
var glovar_regexp *regexp.Regexp
var glovar_mtx sync.Mutex




   

And the init function is supposed to initialize them:
const glovar_regexp_str = `^[a-zA-Z_][a-zA-Z0-9_]*$`

func init() {
glovar_map = make(map[string]**ObjectMo)
glovar_regexp = regexp.MustCompile(glovar_regexp_str)
}

 


For some reason (probably my misunderstanding or bug), the glovar_map 
variable is not initialized like I want it to be. BTW, it is filled in 
RegisterGlobalVariable function 

 
(same file).
And that function gets called in the init function of file globals.go 
 
of the same package.

So I wanted to use a GDB watchpoint on that glovar_map. Sadly, the ELF name 
of that variable is objjvalmo.glovar_map with some internal dot in the name:

~/monimelt % nm bin/monimelt|grep glovar  
0097a7f8 b objvalmo.glovar_map
009990e8 b objvalmo.glovar_mtx
0097a800 b objvalmo.glovar_regexp


and of course when I type watch objvalmo.glovar_map in gdb it does not know 
that variable (because gdb is interpreting the dot as a field separator).

Any tricks for GDB to avoid this confusion? If you need to reproduce my bug 
run bin/monimelt -tiny-dump1 /tmp/foo inside the gdb debugger (in the 
directory  containing README.md). 

FWIW, my Go is 1.8 on Linux/Debian/Sid/x86-64, I'm building with gb, and gdb 
is 7.12 and is reading the /usr/local/go/src/runtime/runtime-gdb.py script.

(I would wish that Go would mangle names with a dollar, not a dot)

Cheers.

-- 
Basile Staynkevitch , France ; email: 
bas...@starynkevitch.net


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: gwenn/gosqlite, go1.8, runtime error: cgo argument has Go pointer to Go pointer

[Basile Starynkevitch]

On Wednesday, March 8, 2017 at 9:40:05 PM UTC+1, Ian Lance Taylor wrote:
>
> On Wed, Mar 8, 2017 at 11:34 AM, Basile Starynkevitch 
> <bas...@starynkevitch.net > wrote: 
> > 
> > On Wednesday, March 8, 2017 at 8:24:26 PM UTC+1, Ian Lance Taylor wrote: 
> >> 
> >> On Wed, Mar 8, 2017 at 7:54 AM, Basile Starynkevitch 
> >> <bas...@starynkevitch.net> wrote: 
> >> I don't have any particular recommendations but I believe that 
> >> github.com/mattn/go-sqlite3 works correctly with cgo.  I know that 
> >> because I sent in patches myself 
> >> 
> >> (
> https://github.com/mattn/go-sqlite3/commit/b76c61051faaf0baf3215e6f811003d98639b702,
>  
>
> >> 
> >> 
> https://github.com/mattn/go-sqlite3/commit/8c66b9cf5ed003dff18db01a55cbd45d35a0990f).
>  
>
> >> 
> > 
> > Thanks Ian. And what about sqlite3_config with SQLITE_CONFIG_LOG ? How 
> do I 
> > call that from Go code? I really want the sqlite log (error) message to 
> > appear somewhere 
> > (just C stderr could be enough). 
>
> I'm sorry, I have no idea.  I've never actually used go-sqlite except 
> to run the tests.  Perhaps someone else here knows more. 
>

Thanks Ian. I managed to write my first cgo thing calling it in my new file 
src/objvalmo/sqlitelog.go 
<https://github.com/bstarynk/monimelt/blob/master/src/objvalmo/sqlitelog.go>  



Basile

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: gwenn/gosqlite, go1.8, runtime error: cgo argument has Go pointer to Go pointer

[Basile Starynkevitch]

On Wednesday, March 8, 2017 at 8:24:26 PM UTC+1, Ian Lance Taylor wrote:
>
> On Wed, Mar 8, 2017 at 7:54 AM, Basile Starynkevitch 
> <bas...@starynkevitch.net > wrote: 
> I don't have any particular recommendations but I believe that 
> github.com/mattn/go-sqlite3 works correctly with cgo.  I know that 
> because I sent in patches myself 
> (
> https://github.com/mattn/go-sqlite3/commit/b76c61051faaf0baf3215e6f811003d98639b702,
>  
>
>
> https://github.com/mattn/go-sqlite3/commit/8c66b9cf5ed003dff18db01a55cbd45d35a0990f).
>  
>
>
>
Thanks Ian. And what about sqlite3_config 
<https://www.sqlite.org/c3ref/config.html> with SQLITE_CONFIG_LOG 
<https://www.sqlite.org/c3ref/c_config_covering_index_scan.html> ? How do I 
call that from Go code? I really want the sqlite log (error) message to 
appear somewhere
(just C stderr could be enough).
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: gwenn/gosqlite, go1.8, runtime error: cgo argument has Go pointer to Go pointer

[Basile Starynkevitch]

On Wednesday, March 8, 2017 at 5:09:06 PM UTC+1, Konstantin Khomoutov wrote:

>
> So if you're not really tied to SQLite as such (and picked it merely 
> because you're familiar with it or because it's a sane go-to choice for 
> single-file structured data persistence solution) other pure-Go 
> solutions exist. 
>

I'm ok with other pure-Go persistence solution, but what are you thinking 
about? A GDBM like thing is perhaps not enough for me 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: gwenn/gosqlite, go1.8, runtime error: cgo argument has Go pointer to Go pointer

[Basile Starynkevitch]

On Wednesday, March 8, 2017 at 2:23:12 PM UTC+1, Tamás Gulácsi wrote:
>
> It's quite straightforward: gwenn/gosqlite has an error, by passing a Go 
> pointer to the C side. This check is in effect since Go1.6.
>
> You should try to refresh your gosqlite library, or find a better 
> maintained one.
>

AFAIU I'm using the latest commit of gosqlite. 

What concrete sqlite glue library for Go do you recommend me, and most 
importantly, how to configure log using  sqlite3_config 
 with SQLITE_CONFIG_LOG? I'm ok 
with any solution.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] gwenn/gosqlite, go1.8, runtime error: cgo argument has Go pointer to Go pointer

[Basile Starynkevitch]

Hello list,

It is about my (github-ed) monimelt  
(MELT monitor, in early pre-alpha stage) specifically about its commit 
44434cd991c 

 
which has some README.md 
 giving build 
instructions; I'm getting a reproducible bug that I don't understand, but I 
am still a newbie in Go.
I'm using go1.8 on Debian/Linux/Sid/x86-64 and I am building with gb 
 (not with go build). The driving idea at first is to 
persist, in sqlite database using some JSON format, the entire heap -of 
some Scheme-like "interpreter"- made of "objects" or "items" and values.

To reproduce my bug, follow my build instructions (essentially, how to use 
gb to build my MELT monitor) then run

./bin/monimelt -final-dump-dir /tmp/


What should happen is that the /tmp/ directory (of course you can 
replace that by any fresh path in /tmp or elsewhere) should be created and 
filled with two (almost empty) sqlite databases.
Actually I am creating temporary files in that directory, with some *random* 
suffix like e.g. +_9l7UhTyyJFx_p6950.tmp (of course when you'll run it 
you'll get *another* random suffix) and I want to rename these temporary 
files when at end of dump.


What is really happening is strange (and reproducible): 
2017/03/08 10:19:35 persist-init installed sqliteerrorlogmo
Monimelt starting pid 6950, Go version go1.8
2017/03/08 10:19:35 monimelt should final dump in /tmp/
2017/03/08 10:19:35 OpenDumperDirectory dirpath=/tmp/ dtempsuf=+
_9l7UhTyyJFx_p6950.tmp
2017/03/08 10:19:35 create_table globflag=true dir=/tmp/
2017/03/08 10:19:35 create_table db=&{0xc420018530 file:/tmp//
monimelt_state.sqlite+_9l7UhTyyJFx_p6950.tmp?mode=rwc=private 0 {0 0} 
[] map[] 0 0 0xc420016480 false map[] map[] 0 0 0 } sql_create_t_params
="CREATE TABLE IF NOT EXISTS t_params \n (par_name VARCHAR(35) PRIMARY KEY 
ASC NOT NULL UNIQUE, \n  par_value TEXT NOT NULL);"
panic: runtime error: cgo argument has Go pointer to Go pointer

goroutine 1 [running]:
github.com/gwenn/gosqlite.(*Conn).ProgressHandler.func2(0x162b488, 
0xc40064, 0xc42000c940)
/home/basile/Documents/monimelt/vendor/src/github.com/gwenn/gosqlite/
trace.go:315 +0x8b
github.com/gwenn/gosqlite.(*Conn).ProgressHandler(0xc42009a640, 0x5833f8, 
0x164, 0x558460, 0xc420018440)
/home/basile/Documents/monimelt/vendor/src/github.com/gwenn/gosqlite/
trace.go:316 +0x117
github.com/gwenn/gosqlite.(*conn).ExecContext(0xc42000e060, 0x8255e0, 
0xc420018440, 0x582e9f, 0x79, 0x851af8, 0x0, 0x0, 0x0, 0x0, ...)
/home/basile/Documents/monimelt/vendor/src/github.com/gwenn/gosqlite/
driver.go:143 +0xcb
database/sql.ctxDriverExec(0x8255e0, 0xc420018440, 0x7fb37ce4e198, 
0xc42000e060, 0x582e9f, 0x79, 0x851af8, 0x0, 0x0, 0x42c0ae, ...)
/usr/local/go/src/database/sql/ctxutil.go:31 +0x28d
database/sql.(*DB).exec.func2()
/usr/local/go/src/database/sql/sql.go:1199 +0x99
database/sql.withLock(0x824ae0, 0xc42001a9a0, 0xc42004b6c0)
/usr/local/go/src/database/sql/sql.go:2545 +0x65
database/sql.(*DB).exec(0xc42009a460, 0x8255e0, 0xc420018440, 0x582e9f, 0x79
, 0x0, 0x0, 0x0, 0x101, 0x0, ...)
/usr/local/go/src/database/sql/sql.go:1200 +0x51c
database/sql.(*DB).ExecContext(0xc42009a460, 0x8255e0, 0xc420018440, 
0x582e9f, 0x79, 0x0, 0x0, 0x0, 0x145, 0x0, ...)
/usr/local/go/src/database/sql/sql.go:1165 +0xbc
database/sql.(*DB).Exec(0xc42009a460, 0x582e9f, 0x79, 0x0, 0x0, 0x0, 0xa0, 
0xc42009a500, 0x0, 0x834a80)
/usr/local/go/src/database/sql/sql.go:1179 +0x85
objvalmo.DumperMo.create_tables(0x0, 0x7ffc2316b893, 0x9, 0xc42000c900, 0x17
, 0xc42009a460, 0xc42009a500, 0x0, 0x0, 0x0, ...)
/home/basile/Documents/monimelt/src/objvalmo/persist.go:208 +0x240
objvalmo.OpenDumperDirectory(0x7ffc2316b893, 0x9, 0x0)
/home/basile/Documents/monimelt/src/objvalmo/persist.go:306 +0xb8b
objvalmo.DumpIntoDirectory(0x7ffc2316b893, 0x9)
/home/basile/Documents/monimelt/src/objvalmo/persist.go:534 +0x39
main.main()
/home/basile/Documents/monimelt/src/monimelt/monimelt.go:38 +0x8df


Now, here is my *partial* understanding of the issue.I suspect I have some 
bug in my code (but then I don't understand where), but it could be 
outside. Maybe it is related to the order of running the various init 
functions (and I don't understand that order, and I don't know how to order 
these initializations). Mayb the bug is elsewhere.

First, *I really need sqlite logging facilities*. So I need the C function 
sqlite3_config  to be called very 
early with SQLITE_CONFIG_LOG 
 and a function 
which justs log verbosely the error. The reason I need sqlite logging is 
that I sometimes am making stupid SQL mistakes, and that logging gives 
verbose enough messages to help me a lot.

So I 

[go-nuts] Re: encoding an integral float like 1.0 as "1.0" in JSON (not "1"), and other JSON questions

[Basile Starynkevitch]

Thanks for all for the help. I managed to commit 4da8b9c7a7d7822ca1e45ce660 


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Go 1.8 compiler SEGV - issue #19323

[Basile Starynkevitch]

On Tuesday, February 28, 2017 at 8:46:47 PM UTC+1, Basile Starynkevitch 
wrote:
>
> Hello All,
>
> I'm still a newbie, and I managed to crash the Go 1.8 compiler 
> (Linux/amd64/Debian/Sid). Probably I have some mistake in my own code (but 
> I am not expecting it to SEGV the compiler)
>
> I've written my first Go report. https://github.com/golang/go/issues/19323
>

Thanks to all contributors to Go. That bug has been identified & fixed! 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] decoding various JSON in specific way (according to first JSON name)

[Basile Starynkevitch]

Hello all,

This question is related to that one 
<https://groups.google.com/d/msg/golang-nuts/nIshrMRrAt0/KSnwExJ4DAAJ> 
(which gives more detailed motivation).

Assume I have some public type 
Person = struct { 
  PersName: string `json:"name"` 
  PersPhone: int `json:"phone"` 
}



I want to unmarshal (i.e. JSON decode) some JSON, with e.g. the following 
rules

a scalar JSON integer is decoded as some Go int64. So Json 123 *⇾* Go 
int64(123) etc

a scalar JSON string is decoded as some Go string. So JSON "abc" *⇾* Go 
string("abc") etc...

a JSON object is decoded according to its *first* name:

   - when that name is "transl" the corresponding value is a number, and 
   the decoding is the closure of that mathematical translation 
   <https://en.wikipedia.org/wiki/Translation_%28geometry%29>. So the JSON { 
   "transl" : 1 }  *⇾* Go closure  func(x int) { return x + 1}  (of course 
   the 1 would actually be the binding of a captured variable y in some 
   closure func(x int) { return x + y }).
   - when that name is "strings" the corresponding value is a JSON array of 
   strings, and its decoding is the slice of such strings. So the JSON { 
   "strings" : [ "a", "b" ] } *⇾* Go strings slice []string{"a","b"}
   - when that name is "name", there is a second name "phone", and its 
   decoding is the appropriate instance of Person. So the JSON { "name" : 
   "John Smith", "phone": 1234567 }
   *⇾* Go Person{PersName:"John Smith", PersPhone: 1234567}

I don't care for other cases or forms of JSON (in particular other forms of 
JSON objects). The decoding (or unmarshaling) could fail (or give some 
other value).

How can I achieve that? I believe that I understand how to code the 
corresponding encoding. I was thinking of using perhaps some dec 
json.Decoder then use dec.Token(), but I am not sure to be able to handle 
the Person case (how to parse two fields). It is strange that not every 
JSON token is actually publicly visible (colons and commas are not).

Of course the actual case is a bit more complex (and not exactly what I am 
explaining above, which is just a simplified example). See my commit 
c8a9212eccbdf2 
<https://github.com/bstarynk/monimelt/commit/c8a9212eccbdf2fa6f74758c2a0c6c75d1dfaa04>

Thanks for reading.

-- 
Basile Starynkevitch <http://starynkevitch.net/Basile/> (France)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] encoding an integral float like 1.0 as "1.0" in JSON (not "1"), and other JSON questions

[Basile Starynkevitch]

On Thursday, March 2, 2017 at 12:01:49 PM UTC+1, Jakob Borg wrote:
>
>
> On 2 Mar 2017, at 18:29, Basile Starynkevitch <bas...@starynkevitch.net 
> > wrote:
>
> I would like that second line to be at least json_emit buf: -1.0 because 
> I need to separate objvalmo.FloatV from objvalmo.IntV types
>
>
> JSON itself doesn't distinguish between integers and floats so it could be 
> a bad fit for your use case. 
>



In theory you are right, json.org don't make the difference. In practice, a 
lot of JSON libraries (e.g. jansson 
<https://jansson.readthedocs.io/en/2.9/apiref.html#number> for C, jsoncpp 
<http://open-source-parsers.github.io/jsoncpp-docs/doxygen/class_json_1_1_value.html>
 
for C++,  YoJson <https://github.com/mjambon/yojson> for Ocaml)  are 
distinguishing them, in particular because they are different types at the 
hardware level.
Separating floats & integers is so common in most JSON implementation that 
I believe it is an implicit part of the JSON spec (I know it is not, but 
JSON parsers are distinguishing numbers with or without decimal points). 
Actually, I can't name any JSON implementation confusing integers and 
floats (but of course there might be, it even looks that YoJson might be 
configured as such, but probably is not by default, because int and float 
are different basic types in Ocaml).

 

> That said, since you have custom types you could implement 
> FloatV.MarshalJSON (or MarshalText) and do the formatting there as you 
> please. 
>

Thanks for the hint. Will try that (not very simple for me to understand 
how).

>
> //jb 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] encoding an integral float like 1.0 as "1.0" in JSON (not "1"), and other JSON questions

[Basile Starynkevitch]

Hello All,

This could be related to issue #6384 


I want to JSON encode a float64 point number always as a JSON float, so 1.0 
should be encoded as 1.0 not as 1 because at decoding time I want to 
distinguish them.
Conceptually, I am persisting in JSON format something which is somehow 
close to some AST, or even to S-expressions (see below for details). So I 
want a float 1.0 to be different of the integer 1.

If you are curious, my exact code is commit 096ec00f53011b2a1b05ce 

 
on github, and when running go test -v objvalmo with my monimelt project 
being last in my GOPATH I'm getting as output (amongst many other lines)

json_emit v=-1.00 of type objvalmo.FloatV
json_emit buf: -1

and I would like that second line to be at least json_emit buf: -1.0 
because I need to separate objvalmo.FloatV from objvalmo.IntV types

I played with the following json/encoding example 
 and 
changed it (by adding some mass being 1.0) to

package main

import (
"encoding/json"
"fmt"
"os"
)

func main() {
h := json.RawMessage(`{"precomputed": true}`)

c := struct {
Header *json.RawMessage `json:"header"`
Body   string   `json:"body"`
Mass   float64`json:"mass"`
}{Header: , Body: "Hello Gophers!", Mass: 1.0}

b, err := json.MarshalIndent(, "", "\t")
if err != nil {
fmt.Println("error:", err)
}
os.Stdout.Write(b)

}




and expected the output to contain "mass" : 1.0 but it still gives a line 
with only  "mass": 1 without any decimal point. Is there any way to change 
that and get that decimal point?


More generally, I don't understand well how should I use JSON facilities in 
Go. IMHO, there are two approaches: one is using go reflection facilities, 
and claim to be able to serialize most arbitrary values. another is not 
using them at all (and this is what I would prefer) by constructing some 
explicit data in memory mimicking JSON format. FWIW, I care a bit about 
performance.

To explain my issues, let us imagine that I want to serialize (and 
unserialize) in JSON format some s-expressions 
 (or some program of a micro 
Scheme or micro Lisp interpreter), with both lists and vectors as composite 
types, with the additional constraint that some symbols should not be 
serialized (and we'll output the null JSON value for them instead) For 
simplicity assume that we have only alphabetical symbols like xy foo bar efg 
Let us suppose that we want to ignore all symbols starting with a vowel  
(so we ignore efg here). so (foo 1 #(2 xy)  (bar "ab" efg -1.0 "cd")) is an 
example of such s-expr.. It is a list of 4 elements. The third element is a 
vector #(2 xy) of 2 components. The 4th and last element is a list (bar 
"ab" efg -1.0 "cd") of 5 elements whose 3rd element is the symbol efg 
which, since starting with a vowel, should be ignored and serialized as null 
and whose 4th element is the floating point -1.0 (not the integer -1). I 
would like to serialize that S-expr value as e.g.
{ "list" : [ { "symb" : "foo" }, 1, { "vect" : [ 2, { "symb" : "xy" } ] }, 
{ "list" : [ { "symb" : "bar" }, null, -1.0, "cd" ] } ] } 


Actually, I am not exacty coding a microScheme. I haved shared objects (of 
*ObjectMo type) and values (of ValueMo type) inside objects (every object 
has a map and a slice of values). I want to persist them in JSON inside an 
Sqlite database, but some objects are transient and non-persistent (so I 
would encode them as null). All objects have a *unique* id like 
_0ECuE7b9XhQ_3HvWl3RGhD6 

Any advices are welcome. In particular, I am a bit afraid of the reflection 
approach (because I want serialization & deserialization to not be too 
slow, and I really need to master the JSON representation).

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] plugin questions....

[Basile Starynkevitch]

On Wednesday, March 1, 2017 at 7:17:42 AM UTC+1, Ian Lance Taylor wrote:
>
> On Tue, Feb 28, 2017 at 12:44 PM, Basile Starynkevitch 
> <bas...@starynkevitch.net > wrote: 
> > Can the packages defined in one plugin be visible from plugins loaded 
> > afterwards? I'm thinking of a dlopen with  RTLD_GLOBAL flag in C 
> parlance. 
>
> I'm not sure this really makes sense in a Go context.  I'm not sure 
> how you would refer to those symbols. 
>

The scenario would be the following:

a plugin *P1* defines the P1_foo () function. *P1* is loaded (and I would 
like the name to be visible afterwards, see below). We can assume that the 
loading program & process does not have any P1_foo in it.

later, *another* plugin *P2* defines the P2_bar() function which calls 
P1_foo(), and I would like that to mean the P1_foo() function inside *P1*. 
It seems that this is not possible currently. (of course loading *P2* 
before loading *P1* should fail). I don't understand why would that not 
make any sense.

Of course, once could play dirty tricks (somehow equivalent to what the 
plugin ELF machinery does internally) like using function variables (that 
is closures) systematically, for example having the main program define a 
map from string to functions and register inside that map at init time of 
the plugin.

Cheers.

-- 
Basile Starynkevitch

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] plugin questions....

[Basile Starynkevitch]

On Tuesday, February 28, 2017 at 10:16:27 PM UTC+1, Ian Lance Taylor wrote:
>
> On Tue, Feb 28, 2017 at 12:44 PM, Basile Starynkevitch 
> <bas...@starynkevitch.net > wrote: 
> > 
> > A few questions and wishes about plugins (Go1.8 Linux/amd64) 
>
> Sorry, but you need to indicate whether you are talking about 
> -buildmode=plugin (an experimental option to build plugins that may be 
> opened by Go programs) or -buildmode=c-shared (a supported option to 
> build plugins that may be opened by non-Go programs). 
>

I'm talking about plugins written in Go for a Go program, so 
-buildmode=plugin
(I'm coding in Go because I don't want to code my multi-thread friendly GC 
in C++ or C)
 

>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] plugin questions....

[Basile Starynkevitch]

Hello All,

A few questions and wishes about plugins (Go1.8 Linux/amd64)

Can a plugin be made of several source files in Go? How should then I 
compile a single plugin myfoo.so from several Go source files ? Can a  
plugin define several packages? 

Can a plugin contain several packages? How can I refer to symbols inside 
packages of plugins? As the usual "packagename.PublicSymbol" passed as 
argument to plugin.Lookup <https://golang.org/pkg/plugin/> ? If yes, that 
should IMHO be documented.

Can one get a plugin handle for the entire program & process (like dlopen(3) 
<http://man7.org/linux/man-pages/man3/dlopen.3.html> does with a NULL 
*filename* argument on Linux)  From my understanding of 
plugin/plugin_dlopen.go <https://golang.org/src/plugin/plugin_dlopen.go> 
this is not the case. I wish it would be the case.

Can the packages defined in one plugin be visible from plugins loaded 
afterwards? I'm thinking of a dlopen with  *RTLD_GLOBAL *flag in C parlance.

I'm interested in having my Go program (monimelt 
<https://github.com/bstarynk/monimelt/> on github, currently Feb2017 in 
pre-alpha stage, so nothing is working) generate at runtime some Go source 
files, compiling them as a plugin (by forking some compilation command at 
runtime), and loading them. BTW, I'm really interested in Go only since its 
plugin abilities so they matter a lot to me.

Is dladdr(3) <http://man7.org/linux/man-pages/man3/dladdr.3.html> available 
in Go? I would like, given a pointer, to get the name of the function at 
the same address (but I guess there could be issues with closures).

Is there any name mangling <https://en.wikipedia.org/wiki/Name_mangling> 
issues the user of Go plugins should be aware of?

Regards.


-- 
Basile Starynkevitch (France)   -   http://starynkevitch.net/Basile/ 



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Go 1.8 compiler SEGV - issue #19323

[Basile Starynkevitch]

Hello All,

I'm still a newbie, and I managed to crash the Go 1.8 compiler 
(Linux/amd64/Debian/Sid). Probably I have some mistake in my own code (but 
I am not expecting it to SEGV the compiler)

I've written my first Go report. https://github.com/golang/go/issues/19323 
Is it complete enough?

For some reason, I might have not been able to upload the faulty program (a 
quite small one). To help Go maintainers, I have attached it here.

Regards

-- 
Basile Starynkevitch (France) http://starynkevitch.net/Basile/


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


monimelt-go-segv.tar.gz
Description: Binary data

[go-nuts] Re: File organization in a github-ed Go project

[Basile Starynkevitch]

On Monday, February 27, 2017 at 3:18:18 PM UTC+1, Basile Starynkevitch 
wrote:
>
>
>
> On Monday, February 27, 2017 at 3:02:24 PM UTC+1, C Banning wrote:
>>
>> Try organizing your project as:
>>
>> monimelt
>>
>> src
>>
>> cmd (or "monimelt", if there's just one app)
>>
>> objvalmo
>>
>> serial 
>>
>>  
>>
>>  Then include $HOME/monimelt in your GOPATH.
>>
>>>
>>>  
> Thanks for the suggestion.
>
> BTW, I did commit in bbc5c3789788 
> <https://github.com/bstarynk/monimelt/commit/bbc5c3789788e507b64776f44f1bc0a698e6e346>
>  
> the change of using directory names same as package names. So I have 
> monimelt/objvalmo/objvalmo.go 
> <https://github.com/bstarynk/monimelt/blob/master/objvalmo/objvalmo.go> & 
> monimelt/serialmo/serialmo.go 
> <https://github.com/bstarynk/monimelt/blob/master/serialmo/serialmo.go> 
> files (implementing packages objvalmo & serialmo respectively)
>
> But does that avoid me to have to use long paths in import directives? 
> Would that allow me to code import "serialmo" instead of import "
> github.com/bstarynk/monimelt/serialmo" in my file objvalmo.go (to ease 
> the forking of my github project)?
>
> Regards
>
> -- 
> Basile Starynkevitch <http://starynkevitch.net/Basile/>
>
>
Actually I did that (adding $HOME/monimelt to my GOPATH)  and now I am 
simply importing "serialmo"

BTW, I am beginning to believe that I could drop the go tool and use gb 
build tool <https://getgb.io/> instead.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: File organization in a github-ed Go project

[Basile Starynkevitch]

On Monday, February 27, 2017 at 3:02:24 PM UTC+1, C Banning wrote:
>
> Try organizing your project as:
>
> monimelt
>
> src
>
> cmd (or "monimelt", if there's just one app)
>
> objvalmo
>
> serial 
>
>  
>
>  Then include $HOME/monimelt in your GOPATH.
>
>>
>>  
Thanks for the suggestion.

BTW, I did commit in bbc5c3789788 
<https://github.com/bstarynk/monimelt/commit/bbc5c3789788e507b64776f44f1bc0a698e6e346>
 
the change of using directory names same as package names. So I have 
monimelt/objvalmo/objvalmo.go 
<https://github.com/bstarynk/monimelt/blob/master/objvalmo/objvalmo.go> & 
monimelt/serialmo/serialmo.go 
<https://github.com/bstarynk/monimelt/blob/master/serialmo/serialmo.go> 
files (implementing packages objvalmo & serialmo respectively)

But does that avoid me to have to use long paths in import directives? 
Would that allow me to code import "serialmo" instead of import 
"github.com/bstarynk/monimelt/serialmo" in my file objvalmo.go (to ease the 
forking of my github project)?

Regards

-- 
Basile Starynkevitch <http://starynkevitch.net/Basile/>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: weak hash for (MongoDb like) data server

[Basile Starynkevitch]

On Thursday, February 16, 2017 at 9:11:08 PM UTC+1, Ian Lance Taylor wrote:

> identified one approach using finalizers.  That approach will work 
> today as Go's garbage collector never moves items.  
>

Thanks Ian for your help. FWIW, I just have coded this objvalmo.go file and 
function 
<https://github.com/bstarynk/monimelt/blob/master/objval/objvalmo.go#L78>
FindOrMakeObjectById 
<https://github.com/bstarynk/monimelt/blob/master/objval/objvalmo.go#L78> 
commit 15a7fbc61e141 
<https://github.com/bstarynk/monimelt/commit/15a7fbc61e141d626554be7ec3de612c5df2885c>
 so 
I hope to have understood your advice correctly.

Regards.

Basile Starynkevitch
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: weak hash for (MongoDb like) data server

[Basile Starynkevitch]

On Thursday, February 16, 2017 at 6:42:05 AM UTC+1, Basile Starynkevitch 
wrote:
>
>
>
> On Wednesday, February 15, 2017 at 9:01:58 PM UTC+1, Tamás Gulácsi wrote:
>
>
>> Why do you need this? 
>> You want the GC do the housekeeping for you, but I'm sure you won't be 
>> happy with the result, as the GC's policy differs from what you await...
>>
>
> What make you believe I won't be happy with the GC's policy. I'm quite 
> open on that. My understanding is that the GC's policy is what I want.
>
>>
>> What kind of eviction policy would you want?
>> Storing the data in an append-only file (or LevelDB), the deleted items 
>> in SQLite, evicting regularly and on close?
>>
>
> Persistence to disk will only happen explicitly at shutdown (process exit) 
> time. So the file aspect don't matter much for my Q1.
>
> So my Q1 is basically: how can I make a weak hash table (in the sense I 
> have described, of having an association from strong keys -pairs of uint64- 
> to weak pointers to items).
>
> BTW, I have found https://github.com/fx5/weakref/blob/master/weakref.go 
but I am not sure to understand how it works (and even if it works). 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: weak hash for (MongoDb like) data server

[Basile Starynkevitch]

On Wednesday, February 15, 2017 at 9:01:58 PM UTC+1, Tamás Gulácsi wrote:


> Why do you need this? 
> You want the GC do the housekeeping for you, but I'm sure you won't be 
> happy with the result, as the GC's policy differs from what you await...
>

What make you believe I won't be happy with the GC's policy. I'm quite open 
on that. My understanding is that the GC's policy is what I want.

>
> What kind of eviction policy would you want?
> Storing the data in an append-only file (or LevelDB), the deleted items in 
> SQLite, evicting regularly and on close?
>

Persistence to disk will only happen explicitly at shutdown (process exit) 
time. So the file aspect don't matter much for my Q1.

So my Q1 is basically: how can I make a weak hash table (in the sense I 
have described, of having an association from strong keys -pairs of uint64- 
to weak pointers to items).
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] weak hash for (MongoDb like) data server

[Basile Starynkevitch]

 ids of its tuple-compoents
  - make a set data value, from the sequence of ids of its set-elements
   - modifying operations, notably
  - adding (or removing) some attribute inside some data item, with a 
  given key data item (and added data value, if relevant)
  - putting some component inside some data item
  - resizing the component vector of some data item
   

And that request would be processed by the server which would return some 
serialization of the resulting data values (with data item references in 
results transmitted by their id)


* Q1: How to associate ids to data items?*


The id of data items is the only way to refer to data items on the protocol 
side. So I need a *weak association* between an id and the data item (if 
any) of that id. I do not want to keep (in some ordinary Go map, for 
example) the association between ids and their data items (otherwise, no 
data item would be garbage collected). I accept the fact that eventually 
(and perhaps quite later), a data item which is not reachable from the root 
data item would be garbage collected.

I was thinking of using unsafe.Pointer-s with runtime.SetFinalizer. A 
simplistic approach could be to use a Go map (module variable) from pairs 
of uint64_t to such unsafe pointers. At data item creation, I would 
register in that map, using the pair of uint64_t-s as key, the pointer of 
that data item suitably converted to an unsafe.Pointer. I would also pass 
to SetFinalizer some finalization routine which would unregister (delete) 
from that map. That scheme would work only if Go garbage collector does not 
move objects (i.e. struct-s). If Go's GC is a conservative GC à la Boehm GC 
<https://www.hboehm.info/gc/>, I'll probably even better "hide" the pointer 
(e.g. by keeping not a Pointer, but some uintptr suitably "hidden" e.g. by 
complementing all its bits à la GC_HIDE_POINTERS 
<https://github.com/ivmai/bdwgc/blob/master/include/gc.h#L1299> in Boehm 
GC). But that scheme would work only if Go's garbage collector is not 
moving values (e.g. struct-s).* Is Go's garbage collector guaranteed to be 
some marking *(not moving or generational copying, à la Ocaml) *one?* FWIW, 
I am familiar with the terminology of the GC handbook 
<http://gchandbook.org/>.

*This Q1 is my main question*. If you answer only to one question, please 
let that be Q1!


*Q2: How to represent attribute associations?*

I'm tempted to use a Go map with data item pointers as keys. But then I 
don't take advantage that ids are hashable. Maybe that is ok. Or should I 
find or implement my self some explicit manually coded hash table?
(I believe I could experiment both, so if you don't have time for Q2, 
please focus on Q1).


*Q3: How to represent data values?*

I'm sure to be able to find out how sum types should be represented in Go 
(a canonical example is of course the AST in the Go compiler itself). But 
if you have time, good hints are welcome. But I prefer answers to Q1 and 
perhaps to Q2 before that Q3.
I might have as additional wish the ability to dynamically load some plugin 
which would represent an additional kind of value (think of floating point, 
or bignums, etc...)

Thanks for reading.
Regards.

PS. Most important question is Q1.
-- 

Basile Starynkevitch http://starynkevitch.net/Basile/   - email is  basile 
at starynkevitch dot net

Bourg La Reine, France






-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.