On Tue, Dec 22, 2020 at 11:21 AM komuW <komu...@gmail.com> wrote: > The bufio package also uses a max buffer of 64KB: > https://go.googlesource.com/go/+/go1.15.6/src/bufio/scan.go#80 >
That limit is for bufio.Scanner. It doesn't have anything to do with bufio.Reader and bufio.Writer, which don't have any particular limits on their buffer sizes. > io.copybuffer on the other hand uses 32KB; > https://go.googlesource.com/go/+/go1.15.6/src/io/io.go#398 > > On Monday, 21 December 2020 at 14:46:21 UTC+3 arn...@gmail.com wrote: > >> @Ben interesting, I did something similar and also ended up with a 64KB >> buffer (seemed like the default of 4KB didn't work very well in my >> context). How did you decide of the buffer size? >> >> Also, there is something that I don't understand. The default buffer >> size works really well for a toy example (like the one I posted in my >> original question), but the improvements are much less dramatic in my >> program, for the same pattern of writing to stdout. I can't work out a >> reason for this yet. >> >> >> On Sunday, 20 December 2020 at 20:31:52 UTC ben...@gmail.com wrote: >> >>> And os.Stdout (and friends) are all regular *os.File objects (which as >>> Jan said, don't buffer). It was non-intuitive to me that stdout didn't >>> buffer by default, because it's such a bad thing for efficiently writing >>> lots of output, but I guess it makes sense when you want terminal output to >>> appear right away. So I realized it made sense, and gives you more control. >>> And it's so easy to wrap it in a bufio.NewWriter() ... Flush() if you need >>> buffering. >>> >>> I ran into this exact same issue when implementing GoAWK ... a 10-line >>> fix gave me a 10x speedup. >>> https://github.com/benhoyt/goawk/commit/60745c3503ba3d99297816f5c7b5364a08ec47ab >>> >>> -Ben >>> >>> On Monday, December 21, 2020 at 12:27:43 AM UTC+13 arn...@gmail.com >>> wrote: >>> >>>> Ah, that is it, thank you! >>>> >>>> On Sunday, 20 December 2020 at 11:06:05 UTC Jan Mercl wrote: >>>> >>>>> On Sun, Dec 20, 2020 at 11:53 AM Arnaud Delobelle <arn...@gmail.com> >>>>> wrote: >>>>> >>>>> > TLDR; a simple test program appears to show that Go's >>>>> (*os.File).Write is 10x slower than C's fputs (on MacOS). >>>>> >>>>> Apples and oranges. fputs buffers, os.File does not. Rewrite the >>>>> benchmark using bufio. >>>>> >>>> -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/f98810f0-9227-4002-a272-d458e21dcf2bn%40googlegroups.com > <https://groups.google.com/d/msgid/golang-nuts/f98810f0-9227-4002-a272-d458e21dcf2bn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CALRNMm2LDpXct-Wi%3DD5J40UroCtDZQc2Y11zokkScVj2H3tTyw%40mail.gmail.com.
