I'm doing similar thing so that a connection can be throttled including to change the throttle in the middle of the request. This can be reused to count outgoing bytes (it was actually a feature that I thought on when designing it, I can't seem to find an implementation of it at the time). Unfortunately my solution works only for http1/* . H2 on the other hand will be much more complex to support due to multiplexing and the fact that some frames are not for any request but for the whole connection. Counting outgoing bytes is still possible per connection, as you said yourself.
The way I did it in nedomi <https://github.com/ironsmile/nedomi> is: - implement net.Listener <https://github.com/ironsmile/nedomi/blob/01acf83d0d615b4f8726e94edf4470985ee176de/utils/netutils/timeout_listener.go#L20> so that I can wrap the new connection in net.Conn implementation <https://github.com/ironsmile/nedomi/blob/01acf83d0d615b4f8726e94edf4470985ee176de/utils/netutils/timeout_conn.go#L26>(this one puts deadlines on each write/read and also is able to throttle the writer <https://github.com/ironsmile/nedomi/blob/01acf83d0d615b4f8726e94edf4470985ee176de/utils/netutils/timeout_conn.go#L96>trough a custom throttle writer <https://github.com/ironsmile/nedomi/blob/01acf83d0d615b4f8726e94edf4470985ee176de/utils/throttle/throttled_writer.go#L29>) the implementation of ReadFrom is because we needed sendfile compatibility - you can take a look at this commit <https://github.com/ironsmile/nedomi/commit/8924019b54b13df926f53fc486a6b6c746993256#diff-98a34a5503a865b1abee3ad0b546e091> for more info - add connstate listener to the server so that you know when a request is closed so that you don't mix reuses of same connection https://github.com/ironsmile/nedomi/blob/01acf83d0d615b4f8726e94edf4470985ee176de/app/app.go#L136 - have a place where to store opened connections (the race condition in Size is optional :P ) https://github.com/ironsmile/nedomi/blob/master/app/connections.go - get the connection out of connection store in the main handler and put in the context (line 54) https://github.com/ironsmile/nedomi/blob/01acf83d0d615b4f8726e94edf4470985ee176de/app/net.go#L46 - Get the connection out of the context in a given later handler and set throttle https://github.com/ironsmile/nedomi/blob/01acf83d0d615b4f8726e94edf4470985ee176de/handler/throttle/throttle.go#L39 You can of course just get the bytes that were written/read when ConnState changes to Inactive and not have the ability to do from inside a handler. Depending on the way your app works you can probably block in the first handler if there is already a request in progress on a given h2 connection in order to be able to differentiate between request (read the doc for ConnState - it won't go to inactive if the connection has an http request in progress so you will need to flush responses to be sure everything was sent before giving the next request a go ahead). I have came up with the hacks above nearly two years ago and there are probably better ways now. I would argue to go look at the implementation in go and try to figure out a way given it - it is the way I have did :). ~MStoykov On Monday, 29 January 2018 23:58:41 UTC+2, Augusto Roman wrote: > > Thanks! > > I'm not willing close the connection on every request -- that's too > painful. > > I don't need an _exact_ count, but I want to be confident that my counts > are really close. Serializing the headers and adding the size of the body > is the closest that I've come up with to do accurate per-request statistics. > > Another idea I had is to have a custom net.Listener that outputs wrapped > net.Conn implementations that will count the number of bytes read/written > to it. It would also track the list of active net.Conns. I think it might > be possible to associate the net.Conn with the request via the local > address using http.LocalAddrContextKey. Then I could count just the number > of bytes per connection and require that each connection is associated with > a given service. I think that would give exactly accurate per-connection > bandwidth, but it would not give per-request bandwidth at all. > > - Augusto > > On Sunday, January 28, 2018 at 10:59:58 PM UTC-7, Kevin Malachowski wrote: >> >> Given TCP connection reuse I don't think the os/kernel will be able to >> tell apart the TCP connections any better than intercepting the net.Conn >> with a custom listener. Especially if you're using anything encrypted, >> staying in Go is going to be necessary for you to get the sort of data you >> need. >> >> You might be able to use your wrapped Listener strategy if you are >> willing to set "Connection: Close" on your incoming connections. I suspect >> that will degrade performance though. You might be able to hack together a >> Listener which is actually able to pool http client connections yet >> convince the http library to Close them when it is done with each request, >> but that sounds hard to do well and would likely to tightly coupled to the >> http package's implementation. >> >> Do you need an exact count? You could probably get a pretty good >> estimation by just serializing the HTTP headers, counting how long that is, >> and adding in the size of the Body. >> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
