Re: [9fans] Go on Plan 9?

[lucio]

> I tried a bootstrapped version on my RPi but it fails with a
> "fork/exec ...  virtual memory allocation failed” error when I try to
> compile anything.  According to stats I have plenty of memory left
> when it runs.  I’m not sure what to make of it.  Any idea if the port
> is complete or if there are additional commits in progress?

I'm sure Richard will respond; I think this is a known problem and has
been addressed.  I seem to recall (not having an rPI of my own) that
the cause lies outside Go.

Lucio.

PS: In passing, I have Plan 9 running - in a limited sense - on
MIPS-64 on my Yeeloong notebook and I have also experimented with
linux/mips64 on the same platform (thank you Cherry for both!).

>From there, I think the next milestones ought to be plan9/mips64 and
exp/shiny.  That would be bliss.

Re: [9fans] Go on Plan 9?

[lucio]

> Skip, isn't the point here that being able to run go binaries
> in Plan 9 on an arm machine is news to most Plan 9 users?

Go seems a little outside the scope of a Plan 9 release and I think it
would take a greater interest by the community to bring it in.  I seem
to recall that Quanstro's 9atom does not implement syscall 53
(nanotime), as a probable example of where the thinking diverges.

It is certainly the case that Go has distinctive philosophy that
differs in place from Plan 9 and I see no reason not to treat the two
as distinct.

Expecting the Plan 9 community to focus on Go would be unreasonable,
in my opinion.  It does not mean that the shift can't take place, but
it should not be forced.

Skip, where in the FS hierarchy do you install the go distribution?
Also, which additional go packages are in general use on Plan 9
platforms and where do they normally get installed?

Lucio.

Re: [9fans] Go on Plan 9?

[Richard Miller]

> I tried a bootstrapped version on my RPi but it fails with a "fork/exec ... 
> virtual memory allocation failed” error when I try to compile anything.

Go needs a lot of virtual memory - it won't even pass the installation test 
suite
if you give it less than a gigabyte.  That was the reason for the change to the
definition in /sys/src/9/bcm/mem.h mentioned earlier:

< #define   USTKTOP 0x2000  /* user segment end +1 
*/
---
> #define   USTKTOP 0x4000  /* user segment end +1 
> */

Are you running a 9pi kernel built with this change?  There are newer kernel 
binaries
in /n/sources/contrib/miller/9pi* with this and other tweaks applied.  If you 
are
using an older pi with 512MB of ram, you'll need to activate swap(8).

The plan9_arm version of go is expected to be in the 1.7 release.  It is already
self hosting: if you look at the builder dashboard in http://build.golang.org
which tracks updates being built and tested on all platforms, the "plan9 arm"
column at the far right is a Raspberry Pi 3 managed by David du Colombier.
It doesn't keep up with every update because a complete build and test suite
run takes a bit over an hour.

Re: [9fans] Go on Plan 9?

[Richard Miller]

> I tried a bootstrapped version on my RPi but it fails with a "fork/exec ... 
> virtual memory allocation failed” error when I try to compile anything.

Go needs a lot of virtual memory - it won't even pass the installation test 
suite
if you give it less than a gigabyte.  That was the reason for the change to the
definition in /sys/src/9/bcm/mem.h mentioned earlier:

< #define   USTKTOP 0x2000  /* user segment end +1 
*/
---
> #define   USTKTOP 0x4000  /* user segment end +1 
> */

Are you running a 9pi kernel built with this change?  There are newer kernel 
binaries
in /n/sources/contrib/miller/9pi* with this and other tweaks applied.  If you 
are
using an older pi with 512MB of ram, you'll need to activate swap(8).

The plan9_arm version of go is expected to be in the 1.7 release.  It is already
self hosting: if you look at the builder dashboard in http://build.golang.org
which tracks updates being built and tested on all platforms, the "plan9 arm"
column near the far right is a Raspberry Pi 3 managed by David du Colombier.
It doesn't keep up with every update because a complete build and test suite
run takes a bit over an hour.

Re: [9fans] Go on Plan 9?

[lucio]

> Next to try on Plan 9: build a linux/s390x binary and find a machine to run
> it on :)

I have certainly done that with linux/386 under Plan 9.  It works like
a charm, even if the compilation is a lot slower than doing it
natively on the target machine (which I could eventually install Go
on).

Lucio.

Re: [9fans] Go on Plan 9?

[Chris McGee]

Thanks Richard for doing the go port to plan9/arm. I was going to start on that 
myself until I found out it was already done. :-)

I didn’t realize that Go was so virtual memory hungry. I wonder why stats 
didn’t show me a large peak of memory consumption before the go compiler died? 
Perhaps it allocates a huge chunk of virtual memory on startup.

I’ll check for that kernel change in my kernel source. If it’s not there I’ll 
recompile and give it a shot. Are these changes going to make it into the 
official kernel source? Any reason why everyone, even non Go users, wouldn’t 
want the changes?

I believe that my rpi only has the 512MB of RAM so I’ll add swap. I didn’t 
realize that Go programs were so heavy weight. What do embedded Go users have 
to do to make things work on other platforms like Linux?

Cheers,
Chris

> On Apr 13, 2016, at 5:10 AM, Richard Miller <9f...@hamnavoe.com> wrote:
> 
>> I tried a bootstrapped version on my RPi but it fails with a "fork/exec ... 
>> virtual memory allocation failed” error when I try to compile anything.
> 
> Go needs a lot of virtual memory - it won't even pass the installation test 
> suite
> if you give it less than a gigabyte.  That was the reason for the change to 
> the
> definition in /sys/src/9/bcm/mem.h mentioned earlier:
> 
> < #define USTKTOP 0x2000  /* user segment end +1 
> */
> ---
>> #define  USTKTOP 0x4000  /* user segment end +1 
>> */
> 
> Are you running a 9pi kernel built with this change?  There are newer kernel 
> binaries
> in /n/sources/contrib/miller/9pi* with this and other tweaks applied.  If you 
> are
> using an older pi with 512MB of ram, you'll need to activate swap(8).
> 
> The plan9_arm version of go is expected to be in the 1.7 release.  It is 
> already
> self hosting: if you look at the builder dashboard in http://build.golang.org
> which tracks updates being built and tested on all platforms, the "plan9 arm"
> column at the far right is a Raspberry Pi 3 managed by David du Colombier.
> It doesn't keep up with every update because a complete build and test suite
> run takes a bit over an hour.
> 
> 

Re: [9fans] Go on Plan 9?

[Richard Miller]

> It won't need a swap file unless the program forces all that to be
> allocated, which it shouldn't,

If you want to get to the satisfying ALL TESTS PASSED message at the end
of the go install+test process, you will need a swap file, even on a
1GB raspberry pi.  Trust me.

Re: [9fans] Go on Plan 9?

[Richard Miller]

> That's insane. Really.

The designer(s) of the test suite had bigger systems in mind,
so there's lots of stuff running concurrently.

Re: [9fans] Go on Plan 9?

[Charles Forsyth]

On 13 April 2016 at 15:42, Charles Forsyth 
wrote:

> On 13 April 2016 at 15:39, Richard Miller <9f...@hamnavoe.com> wrote:
>
>> If you want to get to the satisfying ALL TESTS PASSED message at the end
>> of the go install+test process, you will need a swap file,
>>
>
> That's insane. Really.


More helpfully, it would be useful to find the relevant tests and make them
conditional on configuration.
I'm sure it makes sense to test big memory systems, but it's somewhat
limiting to insist on an Internet of Tanks
instead of Things.

Re: [9fans] Go on Plan 9?

[Richard Miller]

> I didn’t realize that Go was so virtual memory hungry. I wonder why stats 
> didn’t show me a large peak of memory consumption before the go compiler died?

stats -m shows physical memory usage.  Every go program starts by allocating
a huge block of virtual space for its garbage-collected allocation arena.
A lot of that will normally remain unused, so no corresponding physical ram
need be allocated.

> Are these changes going to make it into the official kernel source? Any 
> reason why everyone, even non Go users, wouldn’t want the changes?

Depends what you mean by "official".  The rpi specific changes are all in
/n/sources/contrib/miller/9/bcm, and I will be sending patches for the portable
changes soon.  This is only for reference - it has been some time since any
patches were being applied on /n/sources/plan9.

> I didn’t realize that Go programs were so heavy weight. What do embedded Go 
> users have to do to make things work on other platforms like Linux?

Depends what you mean by "embedded".  The VM allocation is probably less
significant for very small platforms than the size of the runtime library.

cpu% cat smallest.go
package main
func main() {
}
cpu% go build smallest.go
cpu% ls -l smallest
--rwxr-x--x M 3990 miller miller 614356 Apr 13 14:42 smallest
cpu% size smallest
466123t + 1792d + 87712b = 555627   smallest

Re: [9fans] Go on Plan 9?

[Charles Forsyth]

On 13 April 2016 at 14:08, Chris McGee  wrote:

> I believe that my rpi only has the 512MB of RAM so I’ll add swap.


It should be enough to increase the available virtual space by changing
that #define.
It won't need a swap file unless the program forces all that to be
allocated, which it shouldn't,
and if it does, you still won't want to swap since it doesn't work well.

In fact, I've ripped the paging and swap crud out of my own systems. The
time for that was years ago,
and it certainly makes no sense at all on any small device. The code and
data get a lot simpler too.

Re: [9fans] Go on Plan 9?

[Charles Forsyth]

On 13 April 2016 at 15:39, Richard Miller <9f...@hamnavoe.com> wrote:

> If you want to get to the satisfying ALL TESTS PASSED message at the end
> of the go install+test process, you will need a swap file,
>

That's insane. Really.

Re: [9fans] Go on Plan 9?

[lucio]

> If you want to get to the satisfying ALL TESTS PASSED message at the end
> of the go install+test process, you will need a swap file, even on a
> 1GB raspberry pi.  Trust me.

Sounds like a challenge, but I never quite wanted to know whether Plan
9 swap is or isn't broken.  Still, how much swap are we talking about?
It can't be nice on a device like the rPi.

Lucio.

Re: [9fans] Go on Plan 9?

[Chris McGee]

Ah, that makes sense. It’s virtual memory and not the physical memory.

Do you think that your changes to the bcm will make it to 9front? If not, any 
chance I could have the diffs so that I can try merging them in there myself?

Thanks,
Chris

Re: [9fans] Go on Plan 9?

[Kenny Lasse Hoff Levinsen]

I tried that at some point. Got pi2 booting with one core, crashed with 
multiple, but then again, I'm new to having to be that intimate with assembly 
and kernel mode.

I'd suggest trying from scratch to port things, but there are a few 9front 
differences that make it much more than just a diff.

Best regards,
Kenny Levinsen

> On 13. apr. 2016, at 16.00, Chris McGee  wrote:
> 
> Ah, that makes sense. It’s virtual memory and not the physical memory.
> 
> Do you think that your changes to the bcm will make it to 9front? If not, any 
> chance I could have the diffs so that I can try merging them in there myself?
> 
> Thanks,
> Chris

Re: [9fans] Go on Plan 9?

[Richard Miller]

> Still, how much swap are we talking about?

On a 1GB system, the default test suite swaps in only a handful of places.
It's possible to limit the concurrency enough to cut out swapping, but
then it takes longer because there's less opportunity to overlap cpu-bound
tests with file I/O and paging-in of commands.

Re: [9fans] Go on Plan 9?

[Skip Tavakkolian]

I use GOROOT=$home/go to keep up with tip.  after bootstrapping, rebuild
the standard packages and commands:

% GOBIN=$home/bin/386 $home/go/bin/go install -a std cmd

and

% GOARCH=arm go install -a std cmd

then in my profile:

% test -d $home/go/bin/plan9_^$cputype && bind
-a $home/go/bin/plan9_^$cputype /bin  || bind -a $home/go/bin /bin

unfortunately because the treatment of GOBIN is not symmetrical when
$GOARCH == $cputype  vs  $GOARCH != $cputype,  we can't do:

% for (GOARCH in (386 arm)) {
echo GOBIN=$home/bin/$GOARCH $home/go/bin/go install -a std cmd
}


-Skip

On Wed, Apr 13, 2016 at 12:34 AM,  wrote:
>
>
> Skip, where in the FS hierarchy do you install the go distribution?
> Also, which additional go packages are in general use on Plan 9
> platforms and where do they normally get installed?
>
> Lucio.
>
>
>