Re: [go-nuts] bufio usage hurting performance

[Ian Lance Taylor]

On Fri, May 4, 2018 at 7:43 AM, Ankit Gupta  wrote:
>
> I am talking in terms of parsing http request by using
>
> http.ReadRequest method which only accept bufio.Reader
> (https://golang.org/pkg/net/http/#ReadRequest)

Ah.  That may have been a mistake.  It dates back to the initial
implementation of the function, before the first open source release
(https://github.com/golang/go/commit/42b7789a9277e7626e4ddaadcbaefb7689d9d8d4).
If we were to do it today, perhaps we would do it differently.
Unfortunately, we can't change it now without breaking Go 1
compatibility.  I suggest that you add a comment about this to
https://golang.org/issue/5465.  Thanks.

Ian


> On Friday, May 4, 2018 at 8:04:22 PM UTC+5:30, Ian Lance Taylor wrote:
>>
>> On Fri, May 4, 2018 at 5:20 AM, Ankit Gupta  wrote:
>> >
>> > I recently wrote a reverse proxy in Go -
>> > https://github.com/gptankit/serviceq which does load balancing and
>> > queues
>> > failed requests (to dipatch later).
>> > When I compared perf with nginx, I noticed 15-20 ms higher response
>> > times.
>> > So, I timed each function, and got to know that the code spends most
>> > time in
>> > 3 places -
>> >
>> > Read from tcp conn
>> > Save in a buffer
>> > Write to tcp conn
>> >
>> > 'Save' is necessary addition as I need to re-run the requests. For
>> > 'Read'
>> > and 'Write', I use bufio -
>> >
>> > Read:
>> > reader := bufio.NewReader(*httpConn.tcpConn)
>> > req, err := http.ReadRequest(reader)
>> >
>> > Write:
>> > writer := bufio.NewWriter(*httpConn.tcpConn)
>> >
>> > Why does Go impose me to read from bufio.Reader and not directly from
>> > io.Reader? I understand the implication for bigger data but for few
>> > bytes,
>> > managing a buffer seems like an overhead.
>>
>> I'm sorry, I don't understand the question.  Go does not in general
>> require you to use a bufio.Reader.  Where is that requirement coming
>> from?
>>
>>
>> > I am using Go 1.6.
>>
>> That is pretty old now.
>>
>> Ian
>
>
> ::DISCLAIMER::
> 
>
> The contents of this e-mail and any attachments are confidential and
> intended for the named recipient(s) only.E-mail transmission is not
> guaranteed to be secure or error-free as information could be intercepted,
> corrupted,lost, destroyed, arrive late or incomplete, or may contain viruses
> in transmission. The e mail and its contents(with or without referred
> errors) shall therefore not attach any liability on the originator or
> redBus.com. Views or opinions, if any, presented in this email are solely
> those of the author and may not necessarily reflect the views or opinions of
> redBus.com. Any form of reproduction, dissemination, copying, disclosure,
> modification,distribution and / or publication of this message without the
> prior written consent of authorized representative of redbus.com is strictly
> prohibited. If you have received this email in error please delete it and
> notify the sender immediately.Before opening any email and/or attachments,
> please check them for viruses and other defects.
> ::DISCLAIMER::
> 
>
> The contents of this e-mail and any attachments are confidential and
> intended for the named recipient(s) only.E-mail transmission is not
> guaranteed to be secure or error-free as information could be intercepted,
> corrupted,lost, destroyed, arrive late or incomplete, or may contain viruses
> in transmission. The e mail and its contents(with or without referred
> errors) shall therefore not attach any liability on the originator or
> redBus.com. Views or opinions, if any, presented in this email are solely
> those of the author and may not necessarily reflect the views or opinions of
> redBus.com. Any form of reproduction, dissemination, copying, disclosure,
> modification,distribution and / or publication of this message without the
> prior written consent of authorized representative of redbus.com is strictly
> prohibited. If you have received this email in error please delete it and
> notify the sender immediately.Before opening any email and/or attachments,
> please check them for viruses and other defects.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit 

Re: [go-nuts] bufio usage hurting performance

[Ankit Gupta]

I am talking in terms of parsing http request by using

http.ReadRequest method which only accept bufio.Reader 
(https://golang.org/pkg/net/http/#ReadRequest)

On Friday, May 4, 2018 at 8:04:22 PM UTC+5:30, Ian Lance Taylor wrote:
>
> On Fri, May 4, 2018 at 5:20 AM, Ankit Gupta  > wrote: 
> > 
> > I recently wrote a reverse proxy in Go - 
> > https://github.com/gptankit/serviceq which does load balancing and 
> queues 
> > failed requests (to dipatch later). 
> > When I compared perf with nginx, I noticed 15-20 ms higher response 
> times. 
> > So, I timed each function, and got to know that the code spends most 
> time in 
> > 3 places - 
> > 
> > Read from tcp conn 
> > Save in a buffer 
> > Write to tcp conn 
> > 
> > 'Save' is necessary addition as I need to re-run the requests. For 
> 'Read' 
> > and 'Write', I use bufio - 
> > 
> > Read: 
> > reader := bufio.NewReader(*httpConn.tcpConn) 
> > req, err := http.ReadRequest(reader) 
> > 
> > Write: 
> > writer := bufio.NewWriter(*httpConn.tcpConn) 
> > 
> > Why does Go impose me to read from bufio.Reader and not directly from 
> > io.Reader? I understand the implication for bigger data but for few 
> bytes, 
> > managing a buffer seems like an overhead. 
>
> I'm sorry, I don't understand the question.  Go does not in general 
> require you to use a bufio.Reader.  Where is that requirement coming 
> from? 
>
>
> > I am using Go 1.6. 
>
> That is pretty old now. 
>
> Ian 
>

-- 
*::DISCLAIMER::




The contents of this e-mail and any attachments are confidential and 
intended for the named recipient(s) only.E-mail transmission is not 
guaranteed to be secure or error-free as information could be intercepted, 
corrupted,lost, destroyed, arrive late or incomplete, or may contain 
viruses in transmission. The e mail and its contents(with or without 
referred errors) shall therefore not attach any liability on the originator 
or redBus.com. Views or opinions, if any, presented in this email are 
solely those of the author and may not necessarily reflect the views or 
opinions of redBus.com. Any form of reproduction, dissemination, copying, 
disclosure, modification,distribution and / or publication of this message 
without the prior written consent of authorized representative of redbus. 
com is strictly prohibited. If you have received this 
email in error please delete it and notify the sender immediately.Before 
opening any email and/or attachments, please check them for viruses and 
other defects.*

-- 
*::DISCLAIMER::




The contents of this e-mail and any attachments are confidential and 
intended for the named recipient(s) only.E-mail transmission is not 
guaranteed to be secure or error-free as information could be intercepted, 
corrupted,lost, destroyed, arrive late or incomplete, or may contain 
viruses in transmission. The e mail and its contents(with or without 
referred errors) shall therefore not attach any liability on the originator 
or redBus.com. Views or opinions, if any, presented in this email are 
solely those of the author and may not necessarily reflect the views or 
opinions of redBus.com. Any form of reproduction, dissemination, copying, 
disclosure, modification,distribution and / or publication of this message 
without the prior written consent of authorized representative of redbus. 
com is strictly prohibited. If you have received this 
email in error please delete it and notify the sender immediately.Before 
opening any email and/or attachments, please check them for viruses and 
other defects.*

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] bufio usage hurting performance

[Ian Lance Taylor]

On Fri, May 4, 2018 at 5:20 AM, Ankit Gupta  wrote:
>
> I recently wrote a reverse proxy in Go -
> https://github.com/gptankit/serviceq which does load balancing and queues
> failed requests (to dipatch later).
> When I compared perf with nginx, I noticed 15-20 ms higher response times.
> So, I timed each function, and got to know that the code spends most time in
> 3 places -
>
> Read from tcp conn
> Save in a buffer
> Write to tcp conn
>
> 'Save' is necessary addition as I need to re-run the requests. For 'Read'
> and 'Write', I use bufio -
>
> Read:
> reader := bufio.NewReader(*httpConn.tcpConn)
> req, err := http.ReadRequest(reader)
>
> Write:
> writer := bufio.NewWriter(*httpConn.tcpConn)
>
> Why does Go impose me to read from bufio.Reader and not directly from
> io.Reader? I understand the implication for bigger data but for few bytes,
> managing a buffer seems like an overhead.

I'm sorry, I don't understand the question.  Go does not in general
require you to use a bufio.Reader.  Where is that requirement coming
from?


> I am using Go 1.6.

That is pretty old now.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] bufio usage hurting performance

[Ankit Gupta]

I recently wrote a reverse proxy in Go 
- https://github.com/gptankit/serviceq which does load balancing and queues 
failed requests (to dipatch later). 
When I compared perf with nginx, I noticed 15-20 ms higher response times. 
So, I timed each function, and got to know that the code spends most time 
in 3 places - 

Read from tcp conn
Save in a buffer
Write to tcp conn

'Save' is necessary addition as I need to re-run the requests. For 'Read' 
and 'Write', I use bufio - 

Read:
reader := bufio.NewReader(*httpConn.tcpConn)
req, err := http.ReadRequest(reader)

Write:
writer := bufio.NewWriter(*httpConn.tcpConn)

Why does Go impose me to read from bufio.Reader and not directly from 
io.Reader? I understand the implication for bigger data but for few bytes, 
managing a buffer seems like an overhead.

I am using Go 1.6.

-- 
*::DISCLAIMER::




The contents of this e-mail and any attachments are confidential and 
intended for the named recipient(s) only.E-mail transmission is not 
guaranteed to be secure or error-free as information could be intercepted, 
corrupted,lost, destroyed, arrive late or incomplete, or may contain 
viruses in transmission. The e mail and its contents(with or without 
referred errors) shall therefore not attach any liability on the originator 
or redBus.com. Views or opinions, if any, presented in this email are 
solely those of the author and may not necessarily reflect the views or 
opinions of redBus.com. Any form of reproduction, dissemination, copying, 
disclosure, modification,distribution and / or publication of this message 
without the prior written consent of authorized representative of redbus. 
com is strictly prohibited. If you have received this 
email in error please delete it and notify the sender immediately.Before 
opening any email and/or attachments, please check them for viruses and 
other defects.*

-- 
*::DISCLAIMER::




The contents of this e-mail and any attachments are confidential and 
intended for the named recipient(s) only.E-mail transmission is not 
guaranteed to be secure or error-free as information could be intercepted, 
corrupted,lost, destroyed, arrive late or incomplete, or may contain 
viruses in transmission. The e mail and its contents(with or without 
referred errors) shall therefore not attach any liability on the originator 
or redBus.com. Views or opinions, if any, presented in this email are 
solely those of the author and may not necessarily reflect the views or 
opinions of redBus.com. Any form of reproduction, dissemination, copying, 
disclosure, modification,distribution and / or publication of this message 
without the prior written consent of authorized representative of redbus. 
com is strictly prohibited. If you have received this 
email in error please delete it and notify the sender immediately.Before 
opening any email and/or attachments, please check them for viruses and 
other defects.*

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.