Hi
I was under this impression, too I think for my purposes is Better to use a
TLV based stream, also because I am familiar with TLVs. Which means, a
naturally delimited syntax, as you said.
Thanks for the clarification on tcp read behavior.
Thank
On 20 May 2017 17:39, "Jesper Louis Andersen" <
jesper.louis.ander...@gmail.com> wrote:
Hi,
TCP is a stream-oriented protocol. When you send your 1516 bytes, the
receiver may read anything from 0 to 1516 bytes when they execute the read.
The reason is that the operating system kernels in both ends are free to
"chop up" the stream at any point. Furthermore, transmission errors and
retransmissions may also affect what bytes you receive. If you attempt to
use TCP as a message-oriented protocol it will fail eventually. Note that
it could work in a test setting on localhost (where the kernel skips a lot
of stuff and has a large MTU set), while failing once you hit the real
world.
If you want the guarantee you received a full message, you will have to
consume data and then check if you have enough data for completing the
message. Otherwise you loop again and buffer up more data from the stream.
Two usual solutions which are fairly common:
1. Use framing. Prefix your message with a size, for instance a 4 byte
header, big endian 32 bit unsigned integer. Then read until you have that
many bytes in your buffer before continuing processing. The 4 byte overhead
is usually negligible, unless you are sending very short messages. In a
1516 byte message, the overhead is close to 0.2%.
2. Use a format with a natural boundary. You read into a buffer and then
attempt a decode. If the decode succeeds, you pass the message on and
continue with the rest of the stream. If the decode fails, you request more
bytes. Knowing how many more bytes you need allow for a nice optimization
here. For instance, a JSON file is self-delimiting provided all your values
are wrapped in either [] or {}. You simply scan for the end ] or } while
keeping track of what you already parsed in a stack. A protobuf encoding is
not.
Personal subjective advice: Use method 1 above and introduce framing. Don't
try to pack the integer in a variable-width encoding, just make it easy for
the system to handle the integer. More bandwidth is far cheaper than more
CPU or lower energy usage. Read 4 bytes, and determine how many bytes you
need to read. Keep asking the kernel for streamed bytes until you have this
amount. Then decode the message and start processing the next one.
If you try to avoid the framing, it becomes far harder to implement
correctly for a client since the framing decoder can't be kept separate
from a syntactical/semantic encoder. In my experience, here be dragons and
krakens.
Sinister advice: if you need a byte which encodes the type of a message so
it can be discriminated from others, encode the data in the order
Length/Payload/Type rather than Length/Type/Payload. If the Type is in the
suffix it forces programmers to decode everything into a buffer properly
first which tend to remove some errors common to framing decoders.
On Sat, May 20, 2017 at 5:01 PM <fusi.enrico.ma...@gmail.com> wrote:
> Hello All
>
> first I apologize for my English, I'm not a native speaker.
> I have a question about how golang reads the sockets in tcp.
>
> Imagine I send , using a conn.Write(buffer), a buffer which has the
> (unpredictable) size from 80 Bytes to 1516.
>
> Now, if I do something like this (after creating "ln"):
>
> for {
>> message := make([]byte, 1516)
>> var err error
>> // accept connection on port
>> conn, _ := ln.Accept()
>> mytimeout := 1000
>> mytimeout, _ = strconv.Atoi(cf.GConfig["TimeOut"])
>> Gtimeout := time.Duration(mytimeout) * time.Millisecond
>> conn.SetReadDeadline(time.Now().Add(Gtimeout))
>>
>> _, err = conn.Read(message)
>> if err != nil {
>> conn.Close()
>> continue
>> }
>
> *messageinterpreter(message) // just do something with message*
>
> }
>
>
> what I am afraid is that , when the client is sending i.e 100 chars using
> conn.Write(100chars) , the conn.Read will not understand the size, and not
> stop until it has read 1516 bytes
>
> What is the expected behavior in such a case? I cannot serialize this
> content (which is binary) because I fear the impact of overhead.
>
>
> thanks in advance for any comment.
>
> EFM
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
