Frank Schroeder created THRIFT-2619:
---------------------------------------
Summary: Go lib http transport does not handle EOF correctly
Key: THRIFT-2619
URL: https://issues.apache.org/jira/browse/THRIFT-2619
Project: Thrift
Issue Type: Bug
Components: Go - Library
Affects Versions: 0.9.2
Environment: osx
Reporter: Frank Schroeder
Executing the tests with Go 1.3 fails with the following errors:
{code}
frschroeder@Frank:~/git/apache/thrift/lib/go/thrift (master)$ go test
--- FAIL: TestReadWriteBinaryProtocol (0.00 seconds)
protocol_test.go:220: ReadWriteBool: *thrift.TBinaryProtocol
*thrift.THttpClient "EOF" Error reading bool at index 4: %!q(bool=true)
protocol_test.go:223: ReadWriteBool: index 4
&{%!q(*thrift.THttpClient=&{0xc208060510 0xc20802a4d0 0xc20802a540
map[Content-Type:[application/x-thrift]] 0 0})
%!q(*thrift.THttpClient=&{0xc208060510 0xc20802a4d0 0xc20802a540
map[Content-Type:[application/x-thrift]] 0 0})
%!q(*thrift.THttpClient=&{0xc208060510 0xc20802a4d0 0xc20802a540
map[Content-Type:[application/x-thrift]] 0 0})
%!q(*thrift.THttpClient=&{0xc208060510 0xc20802a4d0 0xc20802a540
map[Content-Type:[application/x-thrift]] 0 0}) %!q(bool=false) %!q(bool=true)
"\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"}
&{%!q(*http.Response=&{200 OK 200 HTTP/1.1 1 1 map[Date:[Thu, 10 Jul 2014
09:03:19 GMT] Content-Length:[10] Content-Type:[application/octet-stream]]
0xc20803c8c0 10 [] false map[] 0xc208028d00 <nil>}) %!q(*url.URL=&{http <nil>
127.0.0.1:40000 }) %!q(*bytes.Buffer=&{[] 0 [0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0] 0}) map["Content-Type":["application/x-thrift"]]
'\x00' '\x00'} %!q(bool=true) != %!q(bool=false)
protocol_test.go:269: ReadWriteByte: *thrift.TBinaryProtocol
*thrift.THttpClient "EOF" Error reading byte at index 6: 'ÿ'
protocol_test.go:272: ReadWriteByte: *thrift.TBinaryProtocol
*thrift.THttpClient 255 != 0
protocol_test.go:269: ReadWriteByte: *thrift.TBinaryProtocol
*thrift.THttpClient "EOF" Error reading byte at index 6: 'ÿ'
protocol_test.go:272: ReadWriteByte: *thrift.TBinaryProtocol
*thrift.THttpClient 255 != 0
--- FAIL: TestReadWriteCompactProtocol (0.00 seconds)
protocol_test.go:220: ReadWriteBool: *thrift.TCompactProtocol
*thrift.THttpClient "EOF" Error reading bool at index 4: %!q(bool=true)
protocol_test.go:223: ReadWriteBool: index 4
&{%!q(*thrift.THttpClient=&{0xc2080d0000 0xc208092150 0xc2080921c0
map[Content-Type:[application/x-thrift]] 0 0})
%!q(*thrift.THttpClient=&{0xc2080d0000 0xc208092150 0xc2080921c0
map[Content-Type:[application/x-thrift]] 0 0}) [] '\x00' "" '\x00'
%!q(bool=false) %!q(bool=false) %!q(bool=false)
"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"}
&{%!q(*http.Response=&{200 OK 200 HTTP/1.1 1 1 map[Date:[Thu, 10 Jul 2014
09:03:19 GMT] Content-Length:[6] Content-Type:[application/octet-stream]]
0xc20803d480 6 [] false map[] 0xc2080ce000 <nil>}) %!q(*url.URL=&{http <nil>
127.0.0.1:40000 }) %!q(*bytes.Buffer=&{[] 0 [0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0] 0}) map["Content-Type":["application/x-thrift"]]
'\x00' '\x00'} %!q(bool=true) != %!q(bool=false)
protocol_test.go:269: ReadWriteByte: *thrift.TCompactProtocol
*thrift.THttpClient "EOF" Error reading byte at index 6: 'ÿ'
protocol_test.go:272: ReadWriteByte: *thrift.TCompactProtocol
*thrift.THttpClient 255 != 0
protocol_test.go:306: ReadWriteI16: *thrift.TCompactProtocol
*thrift.THttpClient "EOF" Error reading int16 at index 7: %!q(int16=-32768)
protocol_test.go:309: ReadWriteI16: *thrift.TCompactProtocol
*thrift.THttpClient -32768 != 0
protocol_test.go:343: ReadWriteI32: *thrift.TCompactProtocol
*thrift.THttpClient "EOF" Error reading int32 at index 8: %!q(int32=-2147483535)
protocol_test.go:346: ReadWriteI32: *thrift.TCompactProtocol
*thrift.THttpClient -2147483535 != 0
protocol_test.go:379: ReadWriteI64: *thrift.TCompactProtocol
*thrift.THttpClient "EOF" Error reading int64 at index 12:
%!q(int64=9223372036854775807)
protocol_test.go:382: ReadWriteI64: *thrift.TCompactProtocol
*thrift.THttpClient %!q(int64=9223372036854775807) != '\x00'
protocol_test.go:379: ReadWriteI64: *thrift.TCompactProtocol
*thrift.THttpClient "EOF" Error reading int64 at index 12:
%!q(int64=9223372036854775807)
protocol_test.go:382: ReadWriteI64: *thrift.TCompactProtocol
*thrift.THttpClient %!q(int64=9223372036854775807) != '\x00'
protocol_test.go:269: ReadWriteByte: *thrift.TCompactProtocol
*thrift.THttpClient "EOF" Error reading byte at index 6: 'ÿ'
protocol_test.go:272: ReadWriteByte: *thrift.TCompactProtocol
*thrift.THttpClient 255 != 0
--- FAIL: TestHttpClient (0.00 seconds)
transport_test.go:86: Transport *thrift.THttpClient cannot read binary
data 2 of total length 4096 from offset 0: EOF
FAIL
exit status 1
FAIL _/Users/frschroeder/git/apache/thrift/lib/go/thrift 0.029s
frschroeder@Frank:~/git/apache/thrift/lib/go/thrift (master)$ git log | head
commit b0350dbc40d3bc442f02bbd5980e2c2b5d83194d
Author: jfarrell <[email protected]>
Date: Wed Jul 9 23:39:34 2014 -0400
Thrift-2601:Fix vagrant to work again for builds again
Client: build process
Patch: jfarrell
Updates debian packaging to work with ubuntu 14.04 deps
{code}
The reason for this is that the http.bodyEOFSignal transport changed its
behavior when reaching EOF. With go 1.2 it sent a nil error on the last byte it
read and with go 1.3 it sends the data and EOF in the error.
According to the documented behavior of an io.Reader both is correct and
callers must handle both conditions.
(http://golang.org/pkg/io/#Reader)
{code}
When Read encounters an error or end-of-file condition after successfully
reading n > 0 bytes, it returns the number of bytes read. It may return the
(non-nil) error from the same call or return the error (and n == 0) from a
subsequent call. An instance of this general case is that a Reader returning a
non-zero number of bytes at the end of the input stream may return either err
== EOF or err == nil. The next Read should return 0, EOF regardless.
Callers should always process the n > 0 bytes returned before considering the
error err. Doing so correctly handles I/O errors that happen after reading some
bytes and also both of the allowed EOF behaviors.
{code}
The code in the Go thrift lib does not check for err == io.EOF and assumes that
err != nil is an error. Therefore, the last chunk of data is not read.
I've created a patch and some tests but I'd appreciate if someone could double
check that what I'm doing there makes sense since we're not using the HTTP
transport. Ran the tests with both Go 1.2 and 1.3.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
