Re: [go-nuts] WebAssembly and Filesystem access

[Raffaele Sena]

tinygo now has a "wasi" target that you can try.



On Fri, Dec 16, 2022 at 2:10 PM 'Kevin Chowski' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> I recently learned that WASI (
> https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-intro.md)
> supports filesystem abstractions directly for WASM code.
>
> Is there an plan to integrate this into Go?
>
> On Saturday, November 5, 2022 at 5:59:08 AM UTC-6 Konstantin Khomoutov
> wrote:
>
>> On Fri, Nov 04, 2022 at 02:08:08PM -0700, 'Kevin Chowski' via golang-nuts
>> wrote:
>>
>> [...]
>> > I am on a project which primarily ships a Go command line interface
>> (CLI),
>> > but we have aspirations of using the wasm compilation mode to also
>> > distribute a simple webapp version of it, while sharing most of the
>> code.
>> [...]
>> > The next big hurdle is the fact that we cache things on disk
>> > for later reuse, so things are failing out-of-the-box any time we
>> attempt
>> > to touch the filesystem. Luckily, in our case, the fact that we are
>> > touching the filesystem is a bit incidental (its used as a cache for
>> making
>> > consecutive CLI invocations faster), and as long as the system was
>> > consistent for the lifetime of the main Go process things will work
>> just
>> > fine.
>> >
>> > We /could/ pass a filesystem object across the codebase, and maybe we
>> will
>> > one day we will anyway for some other reasons, but I'd rather not have
>> to
>> > do that just to get further with this webapp prototype: it's a lot of
>> > tedious plumbing, and there is a nontrivial amount of code to touch.
>> >
>> > So instead I decided to try some global-mutable-at-startup variables
>> for
>> > things like os.OpenFile, os.Remove, etc, that should be easy to cleanup
>> if
>> > we decide not to move forward with the prototype; but even then, there
>> are
>> > plenty of random things I have to do to get this to work. I've spent
>> about
>> > 2 hours on this direction so far and I keep hitting unexpected
>> roadblocks -
>> > thus this email seeking out a new strategy.
>> >
>> > Are there any plans to automatically support filesystems on
>> wasm-compiled
>> > Go applications? Even an ephemeral, in-memory filesystem would
>> basically
>> > solve all of my problems without having to change any code on my end,
>> which
>> > would be nice.
>>
>> I'm not a Go developer, but I'd say it is unlikely due to the combination
>> of
>> these two facts:
>>
>> - The os package was implemented as a sort-of grab bag of many features
>> one expects on a typical OS (hence the name). The fact the filesystem
>> operations were implemented in that package directly and not elsewhere
>> is the direct manifestation of that approach.
>> An environment in which WASM runs in a browser is too different from that
>> provided by a typical OS, so I cannot see how one could sensibly
>> implement
>> in GOOS=js.
>>
>> - Even if some sort of FS emulation were to be implemented for GOOS=js,
>> the question is: which one exactly? Keeping stuff in memory in just one
>> way of doing things. While I'm not a web developer, I know of at least
>> session storage and local storage which provide for two levels of
>> persistence beyond keeping stuff in memory. Which one to pick for
>> implementing stuff from the os package?
>>
>> > In lieu of that, does anyone have any tips about how I could go about
>> doing
>> > this in a better way?
>>
>> If you need a hard-and-fast solution, I'd propose you're trying to
>> abstract
>> your stuff away on a wrong level. I think it would be cleaner to factor
>> out
>> your persistence code into a set of functions "store this stuff" and
>> "load
>> that stuff"; the default implementations would call out to the os
>> package, and
>> the wasm implementation would use either approach you select: keeping
>> things
>> in memory, using local storage etc.
>>
>> I mean, trying to implement and pass around through the whole codebase
>> something like a "filesystem accessor" switchable at runtime is only
>> worth it
>> if your code makes heavy use of the filesystem in nontrivial way. I'd say
>> that
>> it's better to abstract away high-level store/load operations.
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/dd8306ab-75bf-4bb6-a9ac-3c9f778665a7n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: [go-nuts] WebAssembly and Filesystem access

['Kevin Chowski' via golang-nuts]

I recently learned that WASI 
(https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-intro.md) 
supports filesystem abstractions directly for WASM code.

Is there an plan to integrate this into Go?

On Saturday, November 5, 2022 at 5:59:08 AM UTC-6 Konstantin Khomoutov 
wrote:

> On Fri, Nov 04, 2022 at 02:08:08PM -0700, 'Kevin Chowski' via golang-nuts 
> wrote:
>
> [...]
> > I am on a project which primarily ships a Go command line interface 
> (CLI), 
> > but we have aspirations of using the wasm compilation mode to also 
> > distribute a simple webapp version of it, while sharing most of the code.
> [...]
> > The next big hurdle is the fact that we cache things on disk 
> > for later reuse, so things are failing out-of-the-box any time we 
> attempt 
> > to touch the filesystem. Luckily, in our case, the fact that we are 
> > touching the filesystem is a bit incidental (its used as a cache for 
> making 
> > consecutive CLI invocations faster), and as long as the system was 
> > consistent for the lifetime of the main Go process things will work just 
> > fine.
> > 
> > We /could/ pass a filesystem object across the codebase, and maybe we 
> will 
> > one day we will anyway for some other reasons, but I'd rather not have 
> to 
> > do that just to get further with this webapp prototype: it's a lot of 
> > tedious plumbing, and there is a nontrivial amount of code to touch.
> > 
> > So instead I decided to try some global-mutable-at-startup variables for 
> > things like os.OpenFile, os.Remove, etc, that should be easy to cleanup 
> if 
> > we decide not to move forward with the prototype; but even then, there 
> are 
> > plenty of random things I have to do to get this to work. I've spent 
> about 
> > 2 hours on this direction so far and I keep hitting unexpected 
> roadblocks - 
> > thus this email seeking out a new strategy.
> > 
> > Are there any plans to automatically support filesystems on 
> wasm-compiled 
> > Go applications? Even an ephemeral, in-memory filesystem would basically 
> > solve all of my problems without having to change any code on my end, 
> which 
> > would be nice.
>
> I'm not a Go developer, but I'd say it is unlikely due to the combination 
> of
> these two facts:
>
> - The os package was implemented as a sort-of grab bag of many features
> one expects on a typical OS (hence the name). The fact the filesystem
> operations were implemented in that package directly and not elsewhere
> is the direct manifestation of that approach.
> An environment in which WASM runs in a browser is too different from that
> provided by a typical OS, so I cannot see how one could sensibly implement
> in GOOS=js.
>
> - Even if some sort of FS emulation were to be implemented for GOOS=js,
> the question is: which one exactly? Keeping stuff in memory in just one
> way of doing things. While I'm not a web developer, I know of at least
> session storage and local storage which provide for two levels of
> persistence beyond keeping stuff in memory. Which one to pick for
> implementing stuff from the os package?
>
> > In lieu of that, does anyone have any tips about how I could go about 
> doing 
> > this in a better way?
>
> If you need a hard-and-fast solution, I'd propose you're trying to abstract
> your stuff away on a wrong level. I think it would be cleaner to factor out
> your persistence code into a set of functions "store this stuff" and "load
> that stuff"; the default implementations would call out to the os package, 
> and
> the wasm implementation would use either approach you select: keeping 
> things
> in memory, using local storage etc.
>
> I mean, trying to implement and pass around through the whole codebase
> something like a "filesystem accessor" switchable at runtime is only worth 
> it
> if your code makes heavy use of the filesystem in nontrivial way. I'd say 
> that
> it's better to abstract away high-level store/load operations.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/dd8306ab-75bf-4bb6-a9ac-3c9f778665a7n%40googlegroups.com.

Re: [go-nuts] Re: Achieving C/C++/Rust comparable performance with Golang

[Sven Nebel]

Thank you for all the insights!

On Fri, 16 Dec 2022 at 14:59, Rich  wrote:

> Too often people look at one benchmark -- speed in execution to determine
> what the best language is. Is Rust better than Go because it can run a
> benchmark faster?  The problem the original developers were trying to solve
> wasn't that C/C++ or other languages couldn't run fast enough, Ken Thompson
> wrote the original Unix kernel in Assembly after all, then went on to write
> B to make it easier to write for the new kernel. Dennis Ritchie modified B
> which became C and so on.  Google had large codebases meant long
> development times when that code needed to be changed to meet business
> requirements, which resulted in long compile times. So, from epic to user
> stories, to executable code was taking too long.
>
> Go is far more agile.  Epic to user stories, to executable code is much
> shorter giving more time for lab testing.  Of course, no program is ever
> 'finished' upon release they'll be new business requirements, Epics, user
> stories and so on. When looking for a language, raw speed shouldn't be the
> only consideration. Go gives decent speeds, but ask yourself does the
> headache of C/C++ / Rust to get a program that may process each call 500ms
> faster worth it? I've tried C/ C++ / Rust --  Great languages, but I'll
> even use Go in place of Bash because I can get it done faster.
> On Thursday, December 15, 2022 at 1:03:23 AM UTC-5 Jason E. Aten wrote:
>
>> two comments
>>
>> a) use many cores. Suddenly your Go code runs circles around everything
>> else-- i.e. those
>> languages you mention where doing multicore in a pain. The view
>> that other languages are faster comes from an age long ago of single core
>> machines.
>>
>> b) rather than pre-mature optimization, the general wisdom, and my
>> specific heartfelt advice,
>>  is to measure where the time and memory is actually being spent, and
>> then optimize from there.
>>  Generally this will 100x speed your development time, and
>> you may find it is plenty fast already, this is on top of the 10x - 20x
>> development speed over C++ that you
>> already get with Go.  Should tuning be desired subsequently, when it
>> comes to profiling,
>>  the Go tools are so vastly superior to anything else out there, that you
>> can easily find the
>> hot spots and focus your optimization with laser precision. This
>> is much more productive than trying to guess in general and over
>> generalized terms about how a given
>> application could perform under different languages.
>>
>> On Wednesday, December 14, 2022 at 11:57:40 AM UTC-6 Kursk wrote:
>>
>>> Hi,
>>> Go is a high performant garbage collected language, (considered) memory
>>> safe programming language, and as such its performance may not be
>>> comparable with other languages such as C/C++/Rust under certain
>>> application scenarios.
>>>
>>> I was wondering, how much of the Go language could be stretched (or
>>> ignored) to produce as close as possible performance results if for
>>> example, we write a program whose garbage collector have no work to do,
>>> assuming that we can pre-allocate all required heap space, or that we could
>>> somehow work around memory safety control points such as, index boundary
>>> checking, etc.
>>>
>>> - Which one those performance penalty points would exactly be?
>>> - Which ones would be impossible to workaround?
>>> - What instructions/operations would be generally speaking comparable?
>>>
>>> I know this is a very abstract and theoretical question, I am of course
>>> not expecting a conclusive/concrete answer to my questions but just some
>>> general answer, to increase my understanding about language performance
>>> while providing some pointers to continue my research on the topic.
>>>
>>> This will hopefully lead me to a better general understanding on where
>>> to seek for potential optimisation opportunities within the Go language
>>> itself vs scenarios where a port to C/C++/Rust would most likely be a
>>> better option.
>>>
>>> Thanks in advance.
>>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/WBnqwLjzTmE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/9cee8bb2-3c63-4eb8-90df-917c592656a4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: [go-nuts] Why not sync.Mutex is designed as an interface

['Yuchen Xie' via golang-nuts]

That makes sense. Thanks for replying.
regards

On Wednesday, November 9, 2022 at 1:11:35 AM UTC+8 Ian Lance Taylor wrote:

> On Tue, Nov 8, 2022 at 7:55 AM Xie Yuchen  wrote:
> >
> > I checked with the issue about how to refer no-copy and the check of 
> cmd/vet. After checking, I'm curious that why not define mutex as an 
> interface, as interface always copy like a reference, which means users 
> don't worry to copy by value and cause an error.
> >
> > Design by the interface can always copy like a reference and no need to 
> mark it as cannot copy. However, we need to provide a default 
> implementation of the mutex and the construction of mutex is not short and 
> clean anymore.
> >
> > Would like to see how do you think about this confusion.
>
> That would mean that the zero value of sync.Mutex would not be usable.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/adffe982-b486-4187-9731-6147ec0bf436n%40googlegroups.com.

[go-nuts] Re: Achieving C/C++/Rust comparable performance with Golang

[Rich]

Too often people look at one benchmark -- speed in execution to determine 
what the best language is. Is Rust better than Go because it can run a 
benchmark faster?  The problem the original developers were trying to solve 
wasn't that C/C++ or other languages couldn't run fast enough, Ken Thompson 
wrote the original Unix kernel in Assembly after all, then went on to write 
B to make it easier to write for the new kernel. Dennis Ritchie modified B 
which became C and so on.  Google had large codebases meant long 
development times when that code needed to be changed to meet business 
requirements, which resulted in long compile times. So, from epic to user 
stories, to executable code was taking too long. 

Go is far more agile.  Epic to user stories, to executable code is much 
shorter giving more time for lab testing.  Of course, no program is ever 
'finished' upon release they'll be new business requirements, Epics, user 
stories and so on. When looking for a language, raw speed shouldn't be the 
only consideration. Go gives decent speeds, but ask yourself does the 
headache of C/C++ / Rust to get a program that may process each call 500ms 
faster worth it? I've tried C/ C++ / Rust --  Great languages, but I'll 
even use Go in place of Bash because I can get it done faster.
On Thursday, December 15, 2022 at 1:03:23 AM UTC-5 Jason E. Aten wrote:

> two comments
>
> a) use many cores. Suddenly your Go code runs circles around everything 
> else-- i.e. those
> languages you mention where doing multicore in a pain. The view
> that other languages are faster comes from an age long ago of single core 
> machines.
>
> b) rather than pre-mature optimization, the general wisdom, and my 
> specific heartfelt advice,
>  is to measure where the time and memory is actually being spent, and then 
> optimize from there.
>  Generally this will 100x speed your development time, and
> you may find it is plenty fast already, this is on top of the 10x - 20x 
> development speed over C++ that you
> already get with Go.  Should tuning be desired subsequently, when it comes 
> to profiling,
>  the Go tools are so vastly superior to anything else out there, that you 
> can easily find the 
> hot spots and focus your optimization with laser precision. This
> is much more productive than trying to guess in general and over 
> generalized terms about how a given 
> application could perform under different languages.
>
> On Wednesday, December 14, 2022 at 11:57:40 AM UTC-6 Kursk wrote:
>
>> Hi,
>> Go is a high performant garbage collected language, (considered) memory 
>> safe programming language, and as such its performance may not be 
>> comparable with other languages such as C/C++/Rust under certain 
>> application scenarios.
>>
>> I was wondering, how much of the Go language could be stretched (or 
>> ignored) to produce as close as possible performance results if for 
>> example, we write a program whose garbage collector have no work to do, 
>> assuming that we can pre-allocate all required heap space, or that we could 
>> somehow work around memory safety control points such as, index boundary 
>> checking, etc.
>>
>> - Which one those performance penalty points would exactly be?
>> - Which ones would be impossible to workaround?
>> - What instructions/operations would be generally speaking comparable?
>>
>> I know this is a very abstract and theoretical question, I am of course 
>> not expecting a conclusive/concrete answer to my questions but just some 
>> general answer, to increase my understanding about language performance 
>> while providing some pointers to continue my research on the topic.
>>
>> This will hopefully lead me to a better general understanding on where to 
>> seek for potential optimisation opportunities within the Go language itself 
>> vs scenarios where a port to C/C++/Rust would most likely be a better 
>> option.
>>
>> Thanks in advance.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9cee8bb2-3c63-4eb8-90df-917c592656a4n%40googlegroups.com.

Re: [go-nuts] How to disable all go compiler build output compressions?

[Dmitriy P]

It seems you not interact with data variable in main or other places, so go 
automatically drop it

Just add `print(len(data))` or something like this in main function and 
binary instantly grow up

пятница, 16 декабря 2022 г. в 10:15:24 UTC+3, Aurora: 

> By empty, I mean it didn't actually contain any actual data, just zeros.
> That's why it's being compressed that much.
> I embedded the file into the Go code using the standard embed directive.
>
> ```
> package main 
>   
>  import _ "embed" 
>   
>  //go:embed .empty 
>  var data []byte
> ```
> On Friday, 16 December 2022 at 01:15:48 UTC+3:30 kra...@skepticism.us 
> wrote:
>
>> Please clarify what you mean by "embedded an empty 100MB file". What 
>> does "empty" mean and how did you "embed" it? Can you show us an example of 
>> what you're trying to do (obviously replacing the 100MB of "empty" with a 
>> placeholder).
>>
>> On Thu, Dec 15, 2022 at 10:58 AM Aurora  wrote:
>>
>>> I've embedded an empty 100MB file in my Go code, for some testing 
>>> purposes.
>>> When I run 'go build', the binary output would be something around 20MB.
>>>
>>> I want the output binary to be more than 100MB, its real size.
>>>
>>> How's this compression being applied to the binary?
>>> How to disable all such optimisations?
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/344ddfba-aea4-4631-b283-202a967f4b02n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>
>>
>> -- 
>> Kurtis Rader
>> Caretaker of the exceptional canines Junior and Hank
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9798befa-9b87-4640-9e44-84c001ab1aacn%40googlegroups.com.