Re: [go-nuts] Performance issue with os.File.Write

2020-12-23 Thread Matt Harden
On Tue, Dec 22, 2020 at 11:21 AM komuW wrote: > The bufio package also uses a max buffer of 64KB: > https://go.googlesource.com/go/+/go1.15.6/src/bufio/scan.go#80 > That limit is for bufio.Scanner. It doesn't have anything to do with bufio.Reader and bufio.Writer, which don't have any

Re: [go-nuts] Performance issue with os.File.Write

2020-12-22 Thread komuW
The bufio package also uses a max buffer of 64KB: https://go.googlesource.com/go/+/go1.15.6/src/bufio/scan.go#80 io.copybuffer on the other hand uses 32KB; https://go.googlesource.com/go/+/go1.15.6/src/io/io.go#398 On Monday, 21 December 2020 at 14:46:21 UTC+3 arn...@gmail.com wrote: > @Ben

Re: [go-nuts] Performance issue with os.File.Write

2020-12-21 Thread Amnon
This is really more of an OS question than a Go language question. Writing single characters is going to be expensive in any language because you pay the considerable overhead of a system call/user-mode to kernel mode context switch overhead for each character. That is why the C stdio library

Re: [go-nuts] Performance issue with os.File.Write

2020-12-21 Thread Ben Hoyt
> @Ben interesting, I did something similar and also ended up with a 64KB buffer (seemed like the default of 4KB didn't work very well in my context). How did you decide of the buffer size? I forget how I ended up with 64KB. I think I was doing performance testing and a buffer larger than the

Re: [go-nuts] Performance issue with os.File.Write

2020-12-21 Thread Arnaud Delobelle
I agree that it's very sensible this way! My confusion stemmed for a combination of blind spots in both Go and C (in fact it was somewhere in my head that the File API provided unbuffered access, just not accessible at that time!). Thanks all for clarifying. Arnaud On Monday, 21 December

Re: [go-nuts] Performance issue with os.File.Write

2020-12-21 Thread Arnaud Delobelle
@Ben interesting, I did something similar and also ended up with a 64KB buffer (seemed like the default of 4KB didn't work very well in my context). How did you decide of the buffer size? Also, there is something that I don't understand. The default buffer size works really well for a toy

Re: [go-nuts] Performance issue with os.File.Write

2020-12-20 Thread Diego Joss
As Jan said "apples and oranges", in this case comparing *os.Stdout with C File *stdout is not fair. The equivalent of *os.Stdout in C is the filedescriptor 1 (STDOUT macro), and the equivalent of *os.Stdout.Write is write(2) (the syscall), not fwrite or fputs. If you retry your microbenchmark

Re: [go-nuts] Performance issue with os.File.Write

2020-12-20 Thread ben...@gmail.com
And os.Stdout (and friends) are all regular *os.File objects (which as Jan said, don't buffer). It was non-intuitive to me that stdout didn't buffer by default, because it's such a bad thing for efficiently writing lots of output, but I guess it makes sense when you want terminal output to

Re: [go-nuts] Performance issue with os.File.Write

2020-12-20 Thread Arnaud Delobelle
Ah, that is it, thank you! On Sunday, 20 December 2020 at 11:06:05 UTC Jan Mercl wrote: > On Sun, Dec 20, 2020 at 11:53 AM Arnaud Delobelle > wrote: > > > TLDR; a simple test program appears to show that Go's (*os.File).Write > is 10x slower than C's fputs (on MacOS). > > Apples and oranges.

Re: [go-nuts] Performance issue with os.File.Write

2020-12-20 Thread Jan Mercl
On Sun, Dec 20, 2020 at 11:53 AM Arnaud Delobelle wrote: > TLDR; a simple test program appears to show that Go's (*os.File).Write is 10x > slower than C's fputs (on MacOS). Apples and oranges. fputs buffers, os.File does not. Rewrite the benchmark using bufio. -- You received this message

[go-nuts] Performance issue with os.File.Write

2020-12-20 Thread Arnaud Delobelle
Hi there! TLDR; a simple test program appears to show that Go's (*os.File).Write is 10x slower than C's fputs (on MacOS). While doing some benchmarking for my lua implementation in Go [1], I found very big differences between C Lua and and golua for benchmarks that do a lot of output to