Your node server lies about its Content-Length. That's the #1 reason for your issues, lack of truth about the Content-Length.
When you under-send (Content-Length longer then actual response body content), you get the java.io.EOFException: EOF reached while reading from your initial message. When you over-send (Content-Length is shorter then your actual response body content), you get the truncated content in the HttpClient (this behavior follows the http spec) I went ahead and put together a demo project to show this at (no npm, no node, using Java and Jetty) https://github.com/joakime/jdk-http-client-example Note: with the jetty, you can't under-send, you'll get a server error instead. Run ServerMain (in your IDE) - and leave it be. Run ClientMain to test all three scenarios. Example output: *Regular request, server determines Content-Length* request: http://localhost:9090/hello response headers: jdk.incubator.http.ImmutableHeaders@d9a98b4c {content-length=[59], content-type=[text/html;charset=utf-8], date=[Wed, 16 May 2018 21:18:52 GMT], server=[Jetty(9.4.10.v20180503)]} } response to get: <html><body><h1>Heading</h1><p>Some Text</p></body></html> *Server sends BAD Content-Length (under-send)* request: http://localhost:9090/hello?forced-len=40 response headers: jdk.incubator.http.ImmutableHeaders@d9a98af4 {content-length=[40], content-type=[text/html;charset=utf-8], date=[Wed, 16 May 2018 21:18:52 GMT], server=[Jetty(9.4.10.v20180503)]} } response to get: <html><body><h1>Heading</h1><p>Some Text *Server sends BAD Content-Length (over-send)* request: http://localhost:9090/hello?forced-len=70 response headers: jdk.incubator.http.ImmutableHeaders@40425da6 {cache-control=[must-revalidate,no-cache,no-store], content-length=[360], content-type=[text/html;charset=iso-8859-1], date=[Wed, 16 May 2018 21:18:52 GMT], server=[Jetty(9.4.10.v20180503)]} } response to get: <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 500 insufficient content written</title> </head> <body><h2>HTTP ERROR 500</h2> <p>Problem accessing /hello. Reason: <pre> insufficient content written</pre></p><hr><a href=" http://eclipse.org/jetty";>Powered by Jetty:// 9.4.10.v20180503</a><hr/> </body> </html> Process finished with exit code 0 On Wed, May 16, 2018 at 11:24 AM, Simon Roberts < si...@dancingcloudservices.com> wrote: > OK, here goes. > > 1) install node 8.11 from here: https://nodejs.org/ > this should consist (though I could be wrong for mac or windows!) of > expanding the archive, and adding the contained 'bin' directory to your > path. > You should be able to execute the commands: > node --version > (note two minus-signs. Should report v8.11.??) > and > npm -version > (note single minus sign! Might report 5.6?? but might be older, not a big > deal) > > 2) clone this: https://github.com/SimonHGR/RESTserver.git > > 3) in the resulting directory (RESTserver) enter: > npm install > (it'll download a bunch of javascript libraries) > > 4) still in the same directory enter: > npm start > > (it should report Express listening on port 8080) > > You should now be able to do > > curl -v http://localhost:8080/simple.html > > and see: > -------------------------------------------------------------------- > * Trying 127.0.0.1... > * Connected to localhost (127.0.0.1) port 8080 (#0) > > GET /simple.html HTTP/1.1 > > Host: localhost:8080 > > User-Agent: curl/7.47.0 > > Accept: */* > > > < HTTP/1.1 200 OK > < X-Powered-By: Express > < Content-Type: text/html; charset=utf-8 > < Content-Length: 95 > < ETag: W/"5f-zPY21XEeilaiEZvppUEI+1JlLE8" > < Date: Wed, 16 May 2018 16:15:35 GMT > < Connection: keep-alive > < > <html> > <head> > </head> > <body> > <h1>A Heading</h1> > <p>A paragraph</p> > </body> > </html> > * Connection #0 to host localhost left intact > > -------------------------------------------------------------------- > > Notes, as a result of the discussion, I deliberately added code, at line > 56 of server.js, to try to ensure that it's sending \r\n. This code isn't > "normal" in node, but I believe it's irrelevant anyway, as I've already > shown that httpClient is failing before it ever processes the body (perhaps > the headers aren't being sent with \r\n!?) > > Also, I did try, last night, to turn off the HTTP2.0 mode in httpClient > using: > > HttpClient client = HttpClient.newBuilder().version(HttpClient.Version. > HTTP_1_1).build(); > > but that did not change anything. > > I'll also note that when trying my "trivial serer", it reproduced the > problem when it was sending the wrong line-endings. > > Anyway, hopefully you can look at the protocol exchange and see what node > is doing that's wrong. > > If you need any updates to the node server, just holler, I'll be around > and able to put a few minutes into this here and there most of today (at > least up through 3pm Mountain time). > > Cheers > > > On Wed, May 16, 2018 at 9:49 AM Simon Roberts <simon@dancingcloudservices. > com> wrote: > >> Thanks for investigating this, Chris. I will put together a trivial node >> server and instructions on how to set it up (it's delightfully simple) >> Should be much less than an hour, if I can give it my un-distracted >> attention... >> >> On Wed, May 16, 2018 at 8:58 AM Chris Hegarty <chris.hega...@oracle.com> >> wrote: >> >>> Bernd, >>> >>> > On 16 May 2018, at 00:43, Bernd Eckenfels <e...@zusammenkunft.net> >>> wrote: >>> > >>> > ... >>> > For the httpclient code, the following improvements are IMHO possible: >>> > >>> > • As you mentioned the EOF should contain the callsite and not >>> be transorted from a worker thread context. This can either be done by >>> rethrwing the EOF Exception or by actually constructing them in the read(). >>> The same (or arguably even worse) Problem is a „connection refused“ type of >>> exception which also has no clear calcite. >>> >>> Agreed. The synchronous send should recreate exceptions >>> before throwing. >>> >>> I filed https://bugs.openjdk.java.net/browse/JDK-8203298 >>> >>> > • Instead of throwing an EOF exception when the read Buffer >>> exceeds the Body lenth I would return a short read till the last available >>> Byte and only throw at the next read. This way the handler can get all of >>> the partial Body. This is however not a good optimization for >>> BodyHander.asString(). >>> >>> Agreed. >>> >>> I think that the client implementation can provide a more >>> helpful exception detail message, than what it does today. >>> >>> I filed https://bugs.openjdk.java.net/browse/JDK-8203302 >>> >>> While maybe not helpful in the string case, it is important >>> that all response body is delivered to the handler before >>> EOF. The following has been file to ensure that that is the >>> case. >>> >>> I filed https://bugs.openjdk.java.net/browse/JDK-8203303 >>> >>> > It is still unlear why the node.js expresss Server has Problems. For >>> that I think its a good idea to trace it either with Wireshark/tcpdump or >>> Maybe one of the web app Debugging reverse proxies. >>> >>> It would be helpful to us to understand exactly what the >>> problem is here. I can try out node.js myself, or if anyone >>> has some instructions to help set that up it would be >>> helpful. >>> >>> -Chris. >> >> >> >> -- >> Simon Roberts >> (303) 249 3613 >> >> > > -- > Simon Roberts > (303) 249 3613 > >
