Re: EOF excption in HTTP 1.1 server interaction

[Bernd Eckenfels]

Hello Simon,

I am also just a user here, but I certainly agree with you that the sample Code 
should be able to communicate with (well behaved) public web Servers.

So if no TLS or htttp2 is involved, I dont think this is the Problem here.

The following is a test Server I used (baed on yours), it does produce a 
correct response (i.e. no exception) if the Parameter is fail=false and a 
aborted exception if the Parameter fail=true (because it writes one Byte less 
than the Content-length would announce and therefore the Client reads past the 
payload)

In the fail case it does indeed not return a Body result (but that is IMHO ok 
the sample Code must not deal with web Servers which are defect)

For the httpclient code, the following improvements are IMHO possible:

a) As you mentioned the EOF should contain the callsite and not be transorted 
from a worker thread context. This can either be done by rethrwing the EOF 
Exception or by actually constructing them in the read(). The same (or arguably 
even worse) Problem is a „connection refused“ type of exception which also has 
no clear callsite.
b) Instead of throwing an EOF exception when the read Buffer exceeds the Body 
lenth I would return a short read till the last available Byte and only throw 
at the next read. This way the handler can get all of the partial Body. This is 
however not a good optimization for BodyHander.asString().

It is still unlear why the node.js expresss Server has Problems. For that I 
think its a good idea to trace it either with Wireshark/tcpdump or Maybe one of 
the web app Debugging reverse proxies.

Gruss
Bernd

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.net.ServerSocket;
import java.net.Socket;

public class FakeSever
{
static final boolean fail = true;
static final String body = "HeadingSome 
Text";
public static final String[] response = {
 "HTTP/1.1 200 OK",
 "X-Powered-By: Expressd",
 "Content-Type: text/html; charset=utf-8",
 "Content-Length: " + (body.length()+(fail?3:2)),
 "ETag: W/\"3c-CQHFqoSATSxoI5iZHLfu5OeEG3k\"",
 "Date: Tue, 15 May 2018 20:34:49 GMT",
 "Connection: keep-alive",
 "",
 body,
};

public static void main(String[] args) throws Throwable {
try (ServerSocket ss = new ServerSocket(8080);) {
System.out.println("Listening...");
try (Socket s = ss.accept();) {
System.out.println("Accepted " + s);
BufferedReader in = new BufferedReader(new 
InputStreamReader(s.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println("< " + line);
if ("".equals(line)) break;
}
Writer  out = new OutputStreamWriter(s.getOutputStream());
for (String st : response) {
out.write(st + "\r\n");
System.out.println("> " + st);
}
out.flush();
}
}
}
}

-- 
http://bernd.eckenfels.net

Von: Simon Roberts
Gesendet: Mittwoch, 16. Mai 2018 00:02
An: net-dev@openjdk.java.net
Betreff: Re: EOF excption in HTTP 1.1 server interaction

No, HTTP 2.0 is not supported by nodejs (... current LTS release, 
eight-dot-something) and by my fake server. I believe that there might be 
something about the way that node is doing that ignoring that's causing the 
problem.

If there's an easy way for me to dump the entire transaction, and anyone cares 
why the send request fails (this thing **never calls the BodyHandler** in case 
I've not made that clear) then I'm happy to run one more test, if it helps. I 
guess if anyone would like to see it, I can install wireshark. Let me know if 
you want me to do that. (And to be clear, this is now for the benefit of anyone 
*else* who wants to pursue this, not for me, I'm gong to continue learning this 
API against a server that it likes.



On Tue, May 15, 2018 at 3:17 PM Bernd Eckenfels  wrote:
When you talk about HTT/2.0 Upgrades, do you also deal with TLS? You can use a 
network tracer instead of curl to debug the whole exchange.

But you might need to turn on TLS debug to dump the session key (so the 
protocol analyser like wireshark can actually decrypt it)

BTW you added one empty line too much in your Simulator Server now, so it’s 62 
bytes.

Gruss
Bernd
-- 
http://bernd.eckenfels.net

From: net-dev  on behalf of Simon Roberts 

Sent: Tuesday, May 15, 2018 10:47:35 PM
To: net-dev@openjdk.java.net
Subject: Re: EOF excption in HTTP 1.1 server interaction 
 
Well, I give up. I'ts something to do with nodejs (which would seem like a 
popular enough server to be of interest, but whatever) and perhaps the way that 
node responds when 

Re: EOF excption in HTTP 1.1 server interaction

[Simon Roberts]

No, HTTP 2.0 is not supported by nodejs (... current LTS release,
eight-dot-something) and by my fake server. I believe that there might be
something about the way that node is doing that ignoring that's causing the
problem.

If there's an easy way for me to dump the entire transaction, and anyone
cares why the send request fails (this thing **never calls the
BodyHandler** in case I've not made that clear) then I'm happy to run one
more test, if it helps. I guess if anyone would like to see it, I can
install wireshark. Let me know if you want me to do that. (And to be clear,
this is now for the benefit of anyone *else* who wants to pursue this, not
for me, I'm gong to continue learning this API against a server that it
likes.



On Tue, May 15, 2018 at 3:17 PM Bernd Eckenfels 
wrote:

> When you talk about HTT/2.0 Upgrades, do you also deal with TLS? You can
> use a network tracer instead of curl to debug the whole exchange.
>
> But you might need to turn on TLS debug to dump the session key (so the
> protocol analyser like wireshark can actually decrypt it)
>
> BTW you added one empty line too much in your Simulator Server now, so
> it’s 62 bytes.
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
> --
> *From:* net-dev  on behalf of Simon
> Roberts 
> *Sent:* Tuesday, May 15, 2018 10:47:35 PM
> *To:* net-dev@openjdk.java.net
> *Subject:* Re: EOF excption in HTTP 1.1 server interaction
>
> Well, I give up. I'ts something to do with nodejs (which would seem like a
> popular enough server to be of interest, but whatever) and perhaps the way
> that node responds when asked to perform an HTTP2.0 upgrade.
>
> Node generated a response that caused httpClient to fail. I used curl to
> try to extract that response and pasted it into my "fake server". The
> response from my fake server works. Of course, the one remaining difference
> is that I cannot tell how node might be reacting to the upgrade request,
> since curl didn't issue that part of the request. So, I think the problem
> is there.
>
>   public static final String[] response = {
>   "HTTP/1.1 200 OK",
>   "X-Powered-By: Expressd",
>   "Content-Type: text/html; charset=utf-8",
>   "Content-Length: 60",
>   "ETag: W/\"3c-CQHFqoSATSxoI5iZHLfu5OeEG3k\"",
>   "Date: Tue, 15 May 2018 20:34:49 GMT",
>   "Connection: keep-alive",
>   "",
>   "",
>   "HeadingSome Text",
>   ""
>   };
>
>   public static void main(String[] args) throws Throwable {
> var ss = new ServerSocket(8080);
> var s = ss.accept();
> var in = new BufferedReader(new InputStreamReader(s.getInputStream()));
> String line;
> while ((line = in.readLine()) != null) {
>   System.out.println("< " + line);
>   if ("".equals(line)) break;
> }
> var out = new PrintWriter(new OutputStreamWriter(s.getOutputStream()));
> for (var st : response) {
>   out.print(st + "\r\n");
>   System.out.println("> " + st);
> }
> out.flush();
> s.close();
>   }
> }
>
> On Tue, May 15, 2018 at 1:55 PM Simon Roberts <
> si...@dancingcloudservices.com> wrote:
>
>> Wooops, bran failure. JLS 3.10.4 :( Will fix to use \r\n and get back...
>>
>>
>> On Tue, May 15, 2018 at 1:43 PM Bernd Eckenfels 
>> wrote:
>>
>>> Try using out.print(st+“\n\r“); instead. (And Account for the extra
>>> bytes in the body as well or output the last string without the EOLs.
>>>
>>> Gruss
>>> Bernd
>>> --
>>> http://bernd.eckenfels.net
>>> --
>>> *From:* net-dev  on behalf of Simon
>>> Roberts 
>>> *Sent:* Tuesday, May 15, 2018 8:44:08 PM
>>> *To:* net-dev@openjdk.java.net
>>> *Subject:* Re: EOF excption in HTTP 1.1 server interaction
>>>
>>> Thanks for the clarification; as I mentioned, I tried a number of
>>> variations, with and without the "excess" empty lines. I also copied the
>>> output from a curl session with a server that provides a successful
>>> interaction with the client (so the content length etc were all correct).
>>> In every case the *send* failed regardless. I'm finding myself inclined to
>>> believe that something about the way the upgrade to 2.0 request is being
>>> handled is relevant. That said, I'm unsure what line endings I'm sending in
>>> my fake server, but I'm running on Linux, not windows (perhaps that's your
>>> point).
>>>
>>> So that we can finally put to bed one way or the other the suggestion
>>> that this is the server-side's fault, perhaps you could indicate an exact
>>> (minimal) string, that you believe should work in my fake-server? I'm not
>>> an expert in the HTTP specification, so it could be pretty inefficient to
>>> keep cycling round this loop ;)
>>>
>>>
>>>
>>> On Tue, May 15, 2018 at 12:16 PM Bernd Eckenfels 
>>> wrote:
>>>
 Your example 

Re: EOF excption in HTTP 1.1 server interaction

[Bernd Eckenfels]

When you talk about HTT/2.0 Upgrades, do you also deal with TLS? You can use a 
network tracer instead of curl to debug the whole exchange.

But you might need to turn on TLS debug to dump the session key (so the 
protocol analyser like wireshark can actually decrypt it)

BTW you added one empty line too much in your Simulator Server now, so it’s 62 
bytes.

Gruss
Bernd
--
http://bernd.eckenfels.net

From: net-dev  on behalf of Simon Roberts 

Sent: Tuesday, May 15, 2018 10:47:35 PM
To: net-dev@openjdk.java.net
Subject: Re: EOF excption in HTTP 1.1 server interaction

Well, I give up. I'ts something to do with nodejs (which would seem like a 
popular enough server to be of interest, but whatever) and perhaps the way that 
node responds when asked to perform an HTTP2.0 upgrade.

Node generated a response that caused httpClient to fail. I used curl to try to 
extract that response and pasted it into my "fake server". The response from my 
fake server works. Of course, the one remaining difference is that I cannot 
tell how node might be reacting to the upgrade request, since curl didn't issue 
that part of the request. So, I think the problem is there.

  public static final String[] response = {
  "HTTP/1.1 200 OK",
  "X-Powered-By: Expressd",
  "Content-Type: text/html; charset=utf-8",
  "Content-Length: 60",
  "ETag: W/\"3c-CQHFqoSATSxoI5iZHLfu5OeEG3k\"",
  "Date: Tue, 15 May 2018 20:34:49 GMT",
  "Connection: keep-alive",
  "",
  "",
  "HeadingSome Text",
  ""
  };

  public static void main(String[] args) throws Throwable {
var ss = new ServerSocket(8080);
var s = ss.accept();
var in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
  System.out.println("< " + line);
  if ("".equals(line)) break;
}
var out = new PrintWriter(new OutputStreamWriter(s.getOutputStream()));
for (var st : response) {
  out.print(st + "\r\n");
  System.out.println("> " + st);
}
out.flush();
s.close();
  }
}

On Tue, May 15, 2018 at 1:55 PM Simon Roberts 
> wrote:
Wooops, bran failure. JLS 3.10.4 :( Will fix to use \r\n and get back...


On Tue, May 15, 2018 at 1:43 PM Bernd Eckenfels 
> wrote:
Try using out.print(st+“\n\r“); instead. (And Account for the extra bytes in 
the body as well or output the last string without the EOLs.

Gruss
Bernd
--
http://bernd.eckenfels.net

From: net-dev 
> on 
behalf of Simon Roberts 
>
Sent: Tuesday, May 15, 2018 8:44:08 PM
To: net-dev@openjdk.java.net
Subject: Re: EOF excption in HTTP 1.1 server interaction

Thanks for the clarification; as I mentioned, I tried a number of variations, 
with and without the "excess" empty lines. I also copied the output from a curl 
session with a server that provides a successful interaction with the client 
(so the content length etc were all correct). In every case the *send* failed 
regardless. I'm finding myself inclined to believe that something about the way 
the upgrade to 2.0 request is being handled is relevant. That said, I'm unsure 
what line endings I'm sending in my fake server, but I'm running on Linux, not 
windows (perhaps that's your point).

So that we can finally put to bed one way or the other the suggestion that this 
is the server-side's fault, perhaps you could indicate an exact (minimal) 
string, that you believe should work in my fake-server? I'm not an expert in 
the HTTP specification, so it could be pretty inefficient to keep cycling round 
this loop ;)



On Tue, May 15, 2018 at 12:16 PM Bernd Eckenfels 
> wrote:
Your example code writes 2 (emp5) lines more than the Content-Length includes. 
You should also use crlf for the end of http headers (Sample works only on 
Windows)ö

Gruss
Bernd

Gruss
Bernd
--
http://bernd.eckenfels.net

From: net-dev 
> on 
behalf of Simon Roberts 
>
Sent: Tuesday, May 15, 2018 7:22:27 PM
To: net-dev@openjdk.java.net
Subject: EOF excption in HTTP 1.1 server interaction

(Added subject line, sorry, not sure how I missed that in the first place!)

I can pretty much confirm this has nothing to do with content length. I wrote 
the code below to allow experimentation and even as it stands, which I believe 
has a correct content length, and a bunch of 

Re: EOF excption in HTTP 1.1 server interaction

[Simon Roberts]

Well, I give up. I'ts something to do with nodejs (which would seem like a
popular enough server to be of interest, but whatever) and perhaps the way
that node responds when asked to perform an HTTP2.0 upgrade.

Node generated a response that caused httpClient to fail. I used curl to
try to extract that response and pasted it into my "fake server". The
response from my fake server works. Of course, the one remaining difference
is that I cannot tell how node might be reacting to the upgrade request,
since curl didn't issue that part of the request. So, I think the problem
is there.

  public static final String[] response = {
  "HTTP/1.1 200 OK",
  "X-Powered-By: Expressd",
  "Content-Type: text/html; charset=utf-8",
  "Content-Length: 60",
  "ETag: W/\"3c-CQHFqoSATSxoI5iZHLfu5OeEG3k\"",
  "Date: Tue, 15 May 2018 20:34:49 GMT",
  "Connection: keep-alive",
  "",
  "",
  "HeadingSome Text",
  ""
  };

  public static void main(String[] args) throws Throwable {
var ss = new ServerSocket(8080);
var s = ss.accept();
var in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
  System.out.println("< " + line);
  if ("".equals(line)) break;
}
var out = new PrintWriter(new OutputStreamWriter(s.getOutputStream()));
for (var st : response) {
  out.print(st + "\r\n");
  System.out.println("> " + st);
}
out.flush();
s.close();
  }
}

On Tue, May 15, 2018 at 1:55 PM Simon Roberts <
si...@dancingcloudservices.com> wrote:

> Wooops, bran failure. JLS 3.10.4 :( Will fix to use \r\n and get back...
>
>
> On Tue, May 15, 2018 at 1:43 PM Bernd Eckenfels 
> wrote:
>
>> Try using out.print(st+“\n\r“); instead. (And Account for the extra bytes
>> in the body as well or output the last string without the EOLs.
>>
>> Gruss
>> Bernd
>> --
>> http://bernd.eckenfels.net
>> --
>> *From:* net-dev  on behalf of Simon
>> Roberts 
>> *Sent:* Tuesday, May 15, 2018 8:44:08 PM
>> *To:* net-dev@openjdk.java.net
>> *Subject:* Re: EOF excption in HTTP 1.1 server interaction
>>
>> Thanks for the clarification; as I mentioned, I tried a number of
>> variations, with and without the "excess" empty lines. I also copied the
>> output from a curl session with a server that provides a successful
>> interaction with the client (so the content length etc were all correct).
>> In every case the *send* failed regardless. I'm finding myself inclined to
>> believe that something about the way the upgrade to 2.0 request is being
>> handled is relevant. That said, I'm unsure what line endings I'm sending in
>> my fake server, but I'm running on Linux, not windows (perhaps that's your
>> point).
>>
>> So that we can finally put to bed one way or the other the suggestion
>> that this is the server-side's fault, perhaps you could indicate an exact
>> (minimal) string, that you believe should work in my fake-server? I'm not
>> an expert in the HTTP specification, so it could be pretty inefficient to
>> keep cycling round this loop ;)
>>
>>
>>
>> On Tue, May 15, 2018 at 12:16 PM Bernd Eckenfels 
>> wrote:
>>
>>> Your example code writes 2 (emp5) lines more than the Content-Length
>>> includes. You should also use crlf for the end of http headers (Sample
>>> works only on Windows)ö
>>>
>>> Gruss
>>> Bernd
>>>
>>> Gruss
>>> Bernd
>>> --
>>> http://bernd.eckenfels.net
>>> --
>>> *From:* net-dev  on behalf of Simon
>>> Roberts 
>>> *Sent:* Tuesday, May 15, 2018 7:22:27 PM
>>> *To:* net-dev@openjdk.java.net
>>> *Subject:* EOF excption in HTTP 1.1 server interaction
>>>
>>> (Added subject line, sorry, not sure how I missed that in the first
>>> place!)
>>>
>>> I can pretty much confirm this has nothing to do with content length. I
>>> wrote the code below to allow experimentation and even as it stands, which
>>> I believe has a correct content length, and a bunch of other stuff removed
>>> from the response) the client still fails. I also tried it with various
>>> combinations of the response lines commented out, and all of them present,
>>> failure every time.
>>>
>>> If you'd like to suggest the combination that "should" work, I'd be
>>> happy to try it of course.
>>>
>>> public final class Main {
>>>   public static final String[] response = {
>>>   "HTTP/1.1 200 OK",
>>> //  "X-Powered-By: Express",
>>> //  "Content-Type: text/html; charset=utf-8",
>>>   "Content-Length: 58",
>>> //  "ETag: W/\"3a-EwoPOQKsJivlqZA3z/ulngzMv9U\"",
>>> //  "Date: Tue, 15 May 2018 00:18:47 GMT",
>>> //  "Connection: keep-alive",
>>>   "",
>>>   "HeadingSome Text",
>>>   "",
>>>   ""
>>>   };
>>>
>>>   public static void main(String[] args) 

Re: EOF excption in HTTP 1.1 server interaction

[Simon Roberts]

Wooops, bran failure. JLS 3.10.4 :( Will fix to use \r\n and get back...


On Tue, May 15, 2018 at 1:43 PM Bernd Eckenfels 
wrote:

> Try using out.print(st+“\n\r“); instead. (And Account for the extra bytes
> in the body as well or output the last string without the EOLs.
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
> --
> *From:* net-dev  on behalf of Simon
> Roberts 
> *Sent:* Tuesday, May 15, 2018 8:44:08 PM
> *To:* net-dev@openjdk.java.net
> *Subject:* Re: EOF excption in HTTP 1.1 server interaction
>
> Thanks for the clarification; as I mentioned, I tried a number of
> variations, with and without the "excess" empty lines. I also copied the
> output from a curl session with a server that provides a successful
> interaction with the client (so the content length etc were all correct).
> In every case the *send* failed regardless. I'm finding myself inclined to
> believe that something about the way the upgrade to 2.0 request is being
> handled is relevant. That said, I'm unsure what line endings I'm sending in
> my fake server, but I'm running on Linux, not windows (perhaps that's your
> point).
>
> So that we can finally put to bed one way or the other the suggestion that
> this is the server-side's fault, perhaps you could indicate an exact
> (minimal) string, that you believe should work in my fake-server? I'm not
> an expert in the HTTP specification, so it could be pretty inefficient to
> keep cycling round this loop ;)
>
>
>
> On Tue, May 15, 2018 at 12:16 PM Bernd Eckenfels 
> wrote:
>
>> Your example code writes 2 (emp5) lines more than the Content-Length
>> includes. You should also use crlf for the end of http headers (Sample
>> works only on Windows)ö
>>
>> Gruss
>> Bernd
>>
>> Gruss
>> Bernd
>> --
>> http://bernd.eckenfels.net
>> --
>> *From:* net-dev  on behalf of Simon
>> Roberts 
>> *Sent:* Tuesday, May 15, 2018 7:22:27 PM
>> *To:* net-dev@openjdk.java.net
>> *Subject:* EOF excption in HTTP 1.1 server interaction
>>
>> (Added subject line, sorry, not sure how I missed that in the first
>> place!)
>>
>> I can pretty much confirm this has nothing to do with content length. I
>> wrote the code below to allow experimentation and even as it stands, which
>> I believe has a correct content length, and a bunch of other stuff removed
>> from the response) the client still fails. I also tried it with various
>> combinations of the response lines commented out, and all of them present,
>> failure every time.
>>
>> If you'd like to suggest the combination that "should" work, I'd be happy
>> to try it of course.
>>
>> public final class Main {
>>   public static final String[] response = {
>>   "HTTP/1.1 200 OK",
>> //  "X-Powered-By: Express",
>> //  "Content-Type: text/html; charset=utf-8",
>>   "Content-Length: 58",
>> //  "ETag: W/\"3a-EwoPOQKsJivlqZA3z/ulngzMv9U\"",
>> //  "Date: Tue, 15 May 2018 00:18:47 GMT",
>> //  "Connection: keep-alive",
>>   "",
>>   "HeadingSome Text",
>>   "",
>>   ""
>>   };
>>
>>   public static void main(String[] args) throws Throwable {
>> var ss = new ServerSocket(8080);
>> var s = ss.accept();
>> var in = new BufferedReader(new
>> InputStreamReader(s.getInputStream()));
>> String line;
>> while ((line = in.readLine()) != null) {
>>   System.out.println("< " + line);
>>   if ("".equals(line)) break;
>> }
>> var out = new PrintWriter(new
>> OutputStreamWriter(s.getOutputStream()));
>> for (var st : response) {
>>   out.println(st);
>>   System.out.println("> " + st);
>> }
>> out.flush();
>> s.close();
>>
>>   }
>> }
>>
>>
>> On Tue, May 15, 2018 at 9:46 AM Chris Hegarty 
>> wrote:
>>
>>> Simon,
>>>
>>> Only a partial reply, I’ll reply with other details later.
>>>
>>> > On 15 May 2018, at 16:10, Simon Roberts <
>>> si...@dancingcloudservices.com> wrote:
>>> >
>>> > If I understand you correctly, you are saying that the "simple"
>>> version of the code-code by the way that the project's main web page
>>> presents as it's first example--is so simple that it's actually unusable in
>>> any production scenario.
>>>
>>> That is not what I am saying. What I am saying is that the String
>>> handler is a convenience handler that buffers all the response data and
>>> returns it once decoded. In some circumstances it is just not possible to
>>> return, as a String, the response data, if the server behaves incorrectly.
>>> In the scenario you are encountering it appears that the server is
>>> returning too little data. It may not be possible to always decode this
>>> partial data. And even if you do decode this partial data, something
>>> further up the stack is likely to fail, if parsing JSON for example.

Re: EOF excption in HTTP 1.1 server interaction

[Simon Roberts]

I have further tweeked my "fake server" so that it should respond with CRLF
at line endings, and tried it with content lengths of 58, 59, and 60.
Regardless, the httpdlient fails *on the call to send* with each of these
experiments. Are there any additional tests or variations that people
suggest trying?

It seems to me that one of the things that's different between the severs
that the httpClient likes, and the ones it doesnt, is that I'm pretty sure
that the ones it doesn't like utterly ignore the HTTP2 upgrade request.

  public static final String[] response = {
  "HTTP/1.1 200 OK",
  "Content-Type: text/html; charset=utf-8",
  "Content-Length: 60",
  "",
  "HeadingSome Text",
  };

  public static void main(String[] args) throws Throwable {
var ss = new ServerSocket(8080);
var s = ss.accept();
var in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
  System.out.println("< " + line);
  if ("".equals(line)) break;
}
var out = new PrintWriter(new OutputStreamWriter(s.getOutputStream()));
for (var st : response) {
  out.print(st + "0x000D0x000A");
  System.out.println("> " + st);
}
out.flush();
s.close();


On Tue, May 15, 2018 at 12:44 PM Simon Roberts <
si...@dancingcloudservices.com> wrote:

> Thanks for the clarification; as I mentioned, I tried a number of
> variations, with and without the "excess" empty lines. I also copied the
> output from a curl session with a server that provides a successful
> interaction with the client (so the content length etc were all correct).
> In every case the *send* failed regardless. I'm finding myself inclined to
> believe that something about the way the upgrade to 2.0 request is being
> handled is relevant. That said, I'm unsure what line endings I'm sending in
> my fake server, but I'm running on Linux, not windows (perhaps that's your
> point).
>
> So that we can finally put to bed one way or the other the suggestion that
> this is the server-side's fault, perhaps you could indicate an exact
> (minimal) string, that you believe should work in my fake-server? I'm not
> an expert in the HTTP specification, so it could be pretty inefficient to
> keep cycling round this loop ;)
>
>
>
> On Tue, May 15, 2018 at 12:16 PM Bernd Eckenfels 
> wrote:
>
>> Your example code writes 2 (emp5) lines more than the Content-Length
>> includes. You should also use crlf for the end of http headers (Sample
>> works only on Windows)ö
>>
>> Gruss
>> Bernd
>>
>> Gruss
>> Bernd
>> --
>> http://bernd.eckenfels.net
>> --
>> *From:* net-dev  on behalf of Simon
>> Roberts 
>> *Sent:* Tuesday, May 15, 2018 7:22:27 PM
>> *To:* net-dev@openjdk.java.net
>> *Subject:* EOF excption in HTTP 1.1 server interaction
>>
>> (Added subject line, sorry, not sure how I missed that in the first
>> place!)
>>
>> I can pretty much confirm this has nothing to do with content length. I
>> wrote the code below to allow experimentation and even as it stands, which
>> I believe has a correct content length, and a bunch of other stuff removed
>> from the response) the client still fails. I also tried it with various
>> combinations of the response lines commented out, and all of them present,
>> failure every time.
>>
>> If you'd like to suggest the combination that "should" work, I'd be happy
>> to try it of course.
>>
>> public final class Main {
>>   public static final String[] response = {
>>   "HTTP/1.1 200 OK",
>> //  "X-Powered-By: Express",
>> //  "Content-Type: text/html; charset=utf-8",
>>   "Content-Length: 58",
>> //  "ETag: W/\"3a-EwoPOQKsJivlqZA3z/ulngzMv9U\"",
>> //  "Date: Tue, 15 May 2018 00:18:47 GMT",
>> //  "Connection: keep-alive",
>>   "",
>>   "HeadingSome Text",
>>   "",
>>   ""
>>   };
>>
>>   public static void main(String[] args) throws Throwable {
>> var ss = new ServerSocket(8080);
>> var s = ss.accept();
>> var in = new BufferedReader(new
>> InputStreamReader(s.getInputStream()));
>> String line;
>> while ((line = in.readLine()) != null) {
>>   System.out.println("< " + line);
>>   if ("".equals(line)) break;
>> }
>> var out = new PrintWriter(new
>> OutputStreamWriter(s.getOutputStream()));
>> for (var st : response) {
>>   out.println(st);
>>   System.out.println("> " + st);
>> }
>> out.flush();
>> s.close();
>>
>>   }
>> }
>>
>>
>> On Tue, May 15, 2018 at 9:46 AM Chris Hegarty 
>> wrote:
>>
>>> Simon,
>>>
>>> Only a partial reply, I’ll reply with other details later.
>>>
>>> > On 15 May 2018, at 16:10, Simon Roberts <
>>> si...@dancingcloudservices.com> wrote:
>>> >
>>> > If I understand you correctly, you are saying that the "simple"
>>> version of the code-code by the way that the 

Re: EOF excption in HTTP 1.1 server interaction

[Simon Roberts]

Thanks for the clarification; as I mentioned, I tried a number of
variations, with and without the "excess" empty lines. I also copied the
output from a curl session with a server that provides a successful
interaction with the client (so the content length etc were all correct).
In every case the *send* failed regardless. I'm finding myself inclined to
believe that something about the way the upgrade to 2.0 request is being
handled is relevant. That said, I'm unsure what line endings I'm sending in
my fake server, but I'm running on Linux, not windows (perhaps that's your
point).

So that we can finally put to bed one way or the other the suggestion that
this is the server-side's fault, perhaps you could indicate an exact
(minimal) string, that you believe should work in my fake-server? I'm not
an expert in the HTTP specification, so it could be pretty inefficient to
keep cycling round this loop ;)



On Tue, May 15, 2018 at 12:16 PM Bernd Eckenfels 
wrote:

> Your example code writes 2 (emp5) lines more than the Content-Length
> includes. You should also use crlf for the end of http headers (Sample
> works only on Windows)ö
>
> Gruss
> Bernd
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
> --
> *From:* net-dev  on behalf of Simon
> Roberts 
> *Sent:* Tuesday, May 15, 2018 7:22:27 PM
> *To:* net-dev@openjdk.java.net
> *Subject:* EOF excption in HTTP 1.1 server interaction
>
> (Added subject line, sorry, not sure how I missed that in the first place!)
>
> I can pretty much confirm this has nothing to do with content length. I
> wrote the code below to allow experimentation and even as it stands, which
> I believe has a correct content length, and a bunch of other stuff removed
> from the response) the client still fails. I also tried it with various
> combinations of the response lines commented out, and all of them present,
> failure every time.
>
> If you'd like to suggest the combination that "should" work, I'd be happy
> to try it of course.
>
> public final class Main {
>   public static final String[] response = {
>   "HTTP/1.1 200 OK",
> //  "X-Powered-By: Express",
> //  "Content-Type: text/html; charset=utf-8",
>   "Content-Length: 58",
> //  "ETag: W/\"3a-EwoPOQKsJivlqZA3z/ulngzMv9U\"",
> //  "Date: Tue, 15 May 2018 00:18:47 GMT",
> //  "Connection: keep-alive",
>   "",
>   "HeadingSome Text",
>   "",
>   ""
>   };
>
>   public static void main(String[] args) throws Throwable {
> var ss = new ServerSocket(8080);
> var s = ss.accept();
> var in = new BufferedReader(new InputStreamReader(s.getInputStream()));
> String line;
> while ((line = in.readLine()) != null) {
>   System.out.println("< " + line);
>   if ("".equals(line)) break;
> }
> var out = new PrintWriter(new OutputStreamWriter(s.getOutputStream()));
> for (var st : response) {
>   out.println(st);
>   System.out.println("> " + st);
> }
> out.flush();
> s.close();
>
>   }
> }
>
>
> On Tue, May 15, 2018 at 9:46 AM Chris Hegarty 
> wrote:
>
>> Simon,
>>
>> Only a partial reply, I’ll reply with other details later.
>>
>> > On 15 May 2018, at 16:10, Simon Roberts 
>> wrote:
>> >
>> > If I understand you correctly, you are saying that the "simple" version
>> of the code-code by the way that the project's main web page presents as
>> it's first example--is so simple that it's actually unusable in any
>> production scenario.
>>
>> That is not what I am saying. What I am saying is that the String handler
>> is a convenience handler that buffers all the response data and returns it
>> once decoded. In some circumstances it is just not possible to return, as a
>> String, the response data, if the server behaves incorrectly. In the
>> scenario you are encountering it appears that the server is returning too
>> little data. It may not be possible to always decode this partial data. And
>> even if you do decode this partial data, something further up the stack is
>> likely to fail, if parsing JSON for example.
>>
>> My point is that if you want fault tolerance in the case of a misbehaving
>> server there are other ways to achieve that.
>>
>> > I know I cannot use code for production that fails to read any data
>> from a (presumably) merely mis-configured server (a server from which all
>> other tools successfully read data.
>>
>> What do you hope to do with partial data read from the server? If it’s
>> JSON can you decode it, or a gif can you display it?
>>
>> > Did I interpret correctly, or did I miss something? If correctly, I'd
>> be surprised if that doesn't strike you as sub-optimal to the point your
>> pride in your work would want to make it more usable.
>>
>> We do have pride in our work. I am engaging here to try to help you.
>>
>> > It doesn't seem 

EOF excption in HTTP 1.1 server interaction

[Simon Roberts]

(Added subject line, sorry, not sure how I missed that in the first place!)

I can pretty much confirm this has nothing to do with content length. I
wrote the code below to allow experimentation and even as it stands, which
I believe has a correct content length, and a bunch of other stuff removed
from the response) the client still fails. I also tried it with various
combinations of the response lines commented out, and all of them present,
failure every time.

If you'd like to suggest the combination that "should" work, I'd be happy
to try it of course.

public final class Main {
  public static final String[] response = {
  "HTTP/1.1 200 OK",
//  "X-Powered-By: Express",
//  "Content-Type: text/html; charset=utf-8",
  "Content-Length: 58",
//  "ETag: W/\"3a-EwoPOQKsJivlqZA3z/ulngzMv9U\"",
//  "Date: Tue, 15 May 2018 00:18:47 GMT",
//  "Connection: keep-alive",
  "",
  "HeadingSome Text",
  "",
  ""
  };

  public static void main(String[] args) throws Throwable {
var ss = new ServerSocket(8080);
var s = ss.accept();
var in = new BufferedReader(new InputStreamReader(s.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
  System.out.println("< " + line);
  if ("".equals(line)) break;
}
var out = new PrintWriter(new OutputStreamWriter(s.getOutputStream()));
for (var st : response) {
  out.println(st);
  System.out.println("> " + st);
}
out.flush();
s.close();

  }
}


On Tue, May 15, 2018 at 9:46 AM Chris Hegarty 
wrote:

> Simon,
>
> Only a partial reply, I’ll reply with other details later.
>
> > On 15 May 2018, at 16:10, Simon Roberts 
> wrote:
> >
> > If I understand you correctly, you are saying that the "simple" version
> of the code-code by the way that the project's main web page presents as
> it's first example--is so simple that it's actually unusable in any
> production scenario.
>
> That is not what I am saying. What I am saying is that the String handler
> is a convenience handler that buffers all the response data and returns it
> once decoded. In some circumstances it is just not possible to return, as a
> String, the response data, if the server behaves incorrectly. In the
> scenario you are encountering it appears that the server is returning too
> little data. It may not be possible to always decode this partial data. And
> even if you do decode this partial data, something further up the stack is
> likely to fail, if parsing JSON for example.
>
> My point is that if you want fault tolerance in the case of a misbehaving
> server there are other ways to achieve that.
>
> > I know I cannot use code for production that fails to read any data from
> a (presumably) merely mis-configured server (a server from which all other
> tools successfully read data.
>
> What do you hope to do with partial data read from the server? If it’s
> JSON can you decode it, or a gif can you display it?
>
> > Did I interpret correctly, or did I miss something? If correctly, I'd be
> surprised if that doesn't strike you as sub-optimal to the point your pride
> in your work would want to make it more usable.
>
> We do have pride in our work. I am engaging here to try to help you.
>
> > It doesn't seem inappropriate to report a "warning" situation using an
> exception? Would not an aggregate return that includes data, headers,
> status code, ... and warnings be more helpful in this case?
>
> There are other ways to achieve that, but IMO doing so for the String
> handler would not be helpful to the vast majority of Java developers, that
> would not check the carried warning.
>
> > Mis-configured servers (assuming that's the cause) are not uncommon
> around the web.
>
> Sure, but many clients will not be able to operate correctly with such,
> it’s a matter of where and how they fail.
>
> > But perhaps more importantly, whatever the problem is, it is not fixed
> by your code. The error is actually thrown by the *send* call, as I've now
> determined as a result of trying your code. (I modified it very slightly so
> as to complete it, and catch the exception.)
>
> D’oh! Apologies, that’s what I get for sending something without testing
> it more carefully, but you seem to have gotten past it.
>
> -Chris



-- 
Simon Roberts
(303) 249 3613

Re:

[Chris Hegarty]

Simon,

Only a partial reply, I’ll reply with other details later.

> On 15 May 2018, at 16:10, Simon Roberts  
> wrote:
> 
> If I understand you correctly, you are saying that the "simple" version of 
> the code-code by the way that the project's main web page presents as it's 
> first example--is so simple that it's actually unusable in any production 
> scenario.

That is not what I am saying. What I am saying is that the String handler is a 
convenience handler that buffers all the response data and returns it once 
decoded. In some circumstances it is just not possible to return, as a String, 
the response data, if the server behaves incorrectly. In the scenario you are 
encountering it appears that the server is returning too little data. It may 
not be possible to always decode this partial data. And even if you do decode 
this partial data, something further up the stack is likely to fail, if parsing 
JSON for example.

My point is that if you want fault tolerance in the case of a misbehaving 
server there are other ways to achieve that.

> I know I cannot use code for production that fails to read any data from a 
> (presumably) merely mis-configured server (a server from which all other 
> tools successfully read data.

What do you hope to do with partial data read from the server? If it’s JSON can 
you decode it, or a gif can you display it?

> Did I interpret correctly, or did I miss something? If correctly, I'd be 
> surprised if that doesn't strike you as sub-optimal to the point your pride 
> in your work would want to make it more usable.

We do have pride in our work. I am engaging here to try to help you.

> It doesn't seem inappropriate to report a "warning" situation using an 
> exception? Would not an aggregate return that includes data, headers, status 
> code, ... and warnings be more helpful in this case?

There are other ways to achieve that, but IMO doing so for the String handler 
would not be helpful to the vast majority of Java developers, that would not 
check the carried warning.

> Mis-configured servers (assuming that's the cause) are not uncommon around 
> the web.

Sure, but many clients will not be able to operate correctly with such, it’s a 
matter of where and how they fail.

> But perhaps more importantly, whatever the problem is, it is not fixed by 
> your code. The error is actually thrown by the *send* call, as I've now 
> determined as a result of trying your code. (I modified it very slightly so 
> as to complete it, and catch the exception.)

D’oh! Apologies, that’s what I get for sending something without testing it 
more carefully, but you seem to have gotten past it.

-Chris

Re:

[Simon Roberts]

If I understand you correctly, you are saying that the "simple" version of
the code-code by the way that the project's main web page presents as it's
first example--is so simple that it's actually unusable in any production
scenario. I know I cannot use code for production that fails to read any
data from a (presumably) merely mis-configured server (a server from which
all other tools successfully read data. Did I interpret correctly, or did I
miss something? If correctly, I'd be surprised if that doesn't strike you
as sub-optimal to the point your pride in your work would want to make it
more usable.

It doesn't seem inappropriate to report a "warning" situation using an
exception? Would not an aggregate return that includes data, headers,
status code, ... and warnings be more helpful in this case? Mis-configured
servers (assuming that's the cause) are not uncommon around the web.

But perhaps more importantly, whatever the problem is, it is not fixed by
your code. The error is actually thrown by the *send* call, as I've now
determined as a result of trying your code. (I modified it very slightly so
as to complete it, and catch the exception.)

  public static void main(String[] args) /*throws Throwable */ {
HttpClient client = HttpClient.newHttpClient();
URI uri = URI.create("http://192.168.1.102:8080/index.html;);

HttpRequest getRequest = HttpRequest.newBuilder()
.uri(uri)
.GET()
.build();

try {
  HttpResponse response = client.send(getRequest,
  HttpResponse.BodyHandler.asInputStream());
  System.out.println("response: " + response);

  response.headers().firstValue("content-length")
  .ifPresent(cl -> System.out.println("content-length:" + cl));

  InputStream is = response.body();
  byte[] bytes = new byte[64 * 1024];
  int read = 0, count;
  try {
while ((count = is.read(bytes, read, bytes.length - read)) != -1)
  read += count;
  } catch (IOException e) {
System.err.println("Ignoring " + e);
  }
  System.out.println("bytes read: " + read);
  String bodyAsString = new String(bytes, StandardCharsets.UTF_8);
  System.out.println("body: " + bodyAsString);
} catch (Throwable t) {
  System.out.println("Bang: ");
  t.printStackTrace();
}
  }

the output follows (note, this is the *entire* output from the program. So
far as I can see, there is no mention of my code in the exception stack
trace (which seems to be a mistake in itself, as it's rather hard to track
down where an error occurred if it's not even mentioned in the trace). I
for my part will turn my attention to getting going with a different server:

/usr/local/jdk-10.0.1/bin/java
-javaagent:/home/simon/idea/idea-IC-181.4445.78/lib/idea_rt.jar=40188:/home/simon/idea/idea-IC-181.4445.78/bin
-Dfile.encoding=UTF-8 -p /home/simon/IdeaProjects/tentest/target/classes -m
tentest/httpstuff.Smartass
WARNING: Using incubator modules: jdk.incubator.httpclient
Bang:
java.io.EOFException: EOF reached while reading
at
jdk.incubator.httpclient/jdk.incubator.http.Http1AsyncReceiver$Http1TubeSubscriber.onComplete(Http1AsyncReceiver.java:507)
at
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadSubscription.signalCompletion(SocketTube.java:551)
at
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:728)
at
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowTask.run(SocketTube.java:171)
at
jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
at
jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
at
jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
at
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:675)
at
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:829)
at
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:243)
at
jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:769)
at
jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:731)

Process finished with exit code 0


On Tue, May 15, 2018 at 8:38 AM Chris Hegarty 
wrote:

> Simon,
>
> > On 15 May 2018, at 14:47, Simon Roberts 
> wrote:
> >
> > Chris, I'm OK with the idea that the program might be "operating in a
> broken environment”
>
> You may be ok with that, but many folk will not.
>
> > but I find it had to accept the idea that "it's not a crash" as useful.
>
> A recoverable exception 

Re:

[Chris Hegarty]

Simon,

> On 15 May 2018, at 14:47, Simon Roberts  
> wrote:
> 
> Chris, I'm OK with the idea that the program might be "operating in a broken 
> environment”

You may be ok with that, but many folk will not.

> but I find it had to accept the idea that "it's not a crash" as useful.

A recoverable exception in Java are not deemed as a “crash”.

> By the time the exception is thrown, I cannot access the data. Merely telling 
> the caller that there was an inconsistency in the server (which I think we're 
> assuming is true) has rendered the data unreachable.

That depends on your particular usage of the API, for example you could
do something like:

  HttpResponse response = client.send(getRequest,
  HttpResponse.BodyHandler.asInputStream());
  System.out.println("response: " + response);
  response.headers().firstValue("content-length")
  .ifPresent(cl -> System.out.println("content-length:" + cl));
  InputStream is = response.body();
  byte[] bytes = new byte[64*1024];
  int read = 0, count;
  try {
  while ((count = is.read(bytes, read, bytes.length - read)) != -1)
  read += count;
  } catch (IOException e) {
  System.err.println("Ignoring " + e);
  }
  System.out.println("bytes read: " + read);
  String bodyAsString = new String(bytes, StandardCharsets.UTF_8);
  System.out.println("body: " + bodyAsString);


> Since every other test mechanism at my disposal survives whatever inaccuracy 
> the node server is producing, it seems a bit ... well, let's say 
> "unforgiving" that this code treats it as an unrecoverable failure.

The API allows for the whole response, including the body, to be retrieved
completely asynchronously, i.e. through one of the String body handlers.
Or for the response headers to be returned first, then the body pulled, i.e.
through an InputStream for example.

While standardising in JDK 11, we’ve updated the javadoc to make this
more clear, as well as adding other pull handlers. Additionally, a custom
handler can be written, with a little extra work, to inspect headers, etc,
before reading the body. Such a handle will be delivered all the response
data before any potential error.

-Chris. 

Re:

[Simon Roberts]

Chris, I'm OK with the idea that the program might be "operating in a
broken environment" but I find it had to accept the idea that "it's not a
crash" as useful. By the time the exception is thrown, I cannot access the
data. Merely telling the caller that there was an inconsistency in the
server (which I think we're assuming is true) has rendered the data
unreachable.

Since every other test mechanism at my disposal survives whatever
inaccuracy the node server is producing, it seems a bit ... well, let's say
"unforgiving" that this code treats it as an unrecoverable failure.


On Tue, May 15, 2018 at 7:31 AM Chris Hegarty 
wrote:

> Simon,
>
> > On 15 May 2018, at 14:25, Simon Roberts 
> wrote:
> >
> > ...
> > I'm running on Linux, not Mac, and t's just a thrown together node.js
> server. I guess that means the real bug is probably in node, though one
> would think that the Java implementation should be robust enough to at
> least not crash (
>
> To be clear, this is NOT a crash ( in Java terms ).
>
> The exception, " java.io.EOFException: EOF reached while reading”,
> that you are seeing is conveying the message that and unexpected
> end-of-file(stream) has been reached while received data. This
> appears to be correct as per Joacim’s observation.
>
> Silently ignoring, or otherwise not reporting, the unexpected EOF,
> would be incorrect and considered a bug.
>
> -Chris.



-- 
Simon Roberts
(303) 249 3613

Re:

[Simon Roberts]

Ah, wait, the content length thing doesn't seem to be all that certain. I
modified my node server to put a line ending on the output, and it still
reports 59 characters (which is now, presumably correct.) The client still
crashes the same way.

meanwhile, I'll try it with a JAX-RS based server instead.


On Tue, May 15, 2018 at 7:25 AM Simon Roberts <
si...@dancingcloudservices.com> wrote:

> Daniel: Thanks, it did seem improbable, but seemed to be the only common
> factor. But I suspect Joakim's astute observation is more relevant.
>
> I'm running on Linux, not Mac, and t's just a thrown together node.js
> server. I guess that means the real bug is probably in node, though one
> would think that the Java implementation should be robust enough to at
> least not crash (
>
> I'll see if I can a) make it work if I run against say a JAX-RS server,
> and b) file an appropriate bug.
>
> Thanks everyone.
>
>
> On Tue, May 15, 2018 at 5:38 AM Joakim Erdfelt 
> wrote:
>
>> You're server is reporting the wrong Content-Length.
>>
>> "HeadingSome Text" is 59
>> characters
>>
>> But the server said "Content-Length: 58"
>>
>> Also, "Connection: keep-alive" is a HTTP/1.0 concept only, but is being
>> reported on your HTTP/1.1 request/response improperly.
>>
>>
>> On Mon, May 14, 2018 at 7:20 PM, Simon Roberts <
>> si...@dancingcloudservices.com> wrote:
>>
>>>
>>> I have a very simple attempt to use the Java 10 / incubator httpclient
>>> api. It works on "regular" resources, but fails horribly when attempting to
>>> get things from localhost. It fails using 127.0.0.1, the actual
>>> non-loopback IP, and the name localhost. The server is running, and
>>> presenting a very simple web page (I originally tried json, but have since
>>> simplified it attempting to find out what's amiss).
>>>
>>> The code is as simple as I can imagine (and that might be the problem :)
>>>
>>> --
>>> HttpClient client = HttpClient.newHttpClient();
>>> URI uri = URI.create("http://192.168.1.102:8080/index.html;);
>>>
>>> HttpRequest getRequest = HttpRequest.newBuilder()
>>> .uri(uri).GET().build();
>>>
>>> HttpResponse response = client.send(getRequest,
>>> HttpResponse.BodyHandler.asString());
>>> System.out.println("response to get: " + response.body());
>>> --
>>>
>>> The error prints a long stack trace, which I suppose I must post in its
>>> entiretly:
>>>
>>> --
>>>
>>> WARNING: Using incubator modules: jdk.incubator.httpclient
>>> Exception in thread "main" java.io.EOFException: EOF reached while
>>> reading
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.Http1AsyncReceiver$Http1TubeSubscriber.onComplete(Http1AsyncReceiver.java:507)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadSubscription.signalCompletion(SocketTube.java:551)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:728)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowTask.run(SocketTube.java:171)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:675)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:829)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:243)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:769)
>>> at
>>> jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:731)
>>>
>>> Process finished with exit code 1
>>> --
>>>
>>> If I use a curl to make the request of my (node/express) server, I get:
>>>
>>> --
>>> $ curl -v http://localhost:8080/index.html
>>> *   Trying 127.0.0.1...
>>> * Connected to localhost (127.0.0.1) port 8080 (#0)
>>> > GET /index.html HTTP/1.1
>>> > Host: localhost:8080
>>> > User-Agent: curl/7.47.0
>>> > Accept: */*
>>> >
>>> < HTTP/1.1 200 OK
>>> < X-Powered-By: Express
>>> < Content-Type: text/html; charset=utf-8
>>> < Content-Length: 58
>>> < ETag: W/"3a-EwoPOQKsJivlqZA3z/ulngzMv9U"
>>> < 

Re:

[Chris Hegarty]

Simon,

> On 15 May 2018, at 14:25, Simon Roberts  
> wrote:
> 
> ...
> I'm running on Linux, not Mac, and t's just a thrown together node.js server. 
> I guess that means the real bug is probably in node, though one would think 
> that the Java implementation should be robust enough to at least not crash (

To be clear, this is NOT a crash ( in Java terms ).

The exception, " java.io.EOFException: EOF reached while reading”,
that you are seeing is conveying the message that and unexpected
end-of-file(stream) has been reached while received data. This
appears to be correct as per Joacim’s observation.

Silently ignoring, or otherwise not reporting, the unexpected EOF,
would be incorrect and considered a bug.

-Chris.

Re:

[Simon Roberts]

Daniel: Thanks, it did seem improbable, but seemed to be the only common
factor. But I suspect Joakim's astute observation is more relevant.

I'm running on Linux, not Mac, and t's just a thrown together node.js
server. I guess that means the real bug is probably in node, though one
would think that the Java implementation should be robust enough to at
least not crash (

I'll see if I can a) make it work if I run against say a JAX-RS server, and
b) file an appropriate bug.

Thanks everyone.


On Tue, May 15, 2018 at 5:38 AM Joakim Erdfelt 
wrote:

> You're server is reporting the wrong Content-Length.
>
> "HeadingSome Text" is 59
> characters
>
> But the server said "Content-Length: 58"
>
> Also, "Connection: keep-alive" is a HTTP/1.0 concept only, but is being
> reported on your HTTP/1.1 request/response improperly.
>
>
> On Mon, May 14, 2018 at 7:20 PM, Simon Roberts <
> si...@dancingcloudservices.com> wrote:
>
>>
>> I have a very simple attempt to use the Java 10 / incubator httpclient
>> api. It works on "regular" resources, but fails horribly when attempting to
>> get things from localhost. It fails using 127.0.0.1, the actual
>> non-loopback IP, and the name localhost. The server is running, and
>> presenting a very simple web page (I originally tried json, but have since
>> simplified it attempting to find out what's amiss).
>>
>> The code is as simple as I can imagine (and that might be the problem :)
>>
>> --
>> HttpClient client = HttpClient.newHttpClient();
>> URI uri = URI.create("http://192.168.1.102:8080/index.html;);
>>
>> HttpRequest getRequest = HttpRequest.newBuilder()
>> .uri(uri).GET().build();
>>
>> HttpResponse response = client.send(getRequest,
>> HttpResponse.BodyHandler.asString());
>> System.out.println("response to get: " + response.body());
>> --
>>
>> The error prints a long stack trace, which I suppose I must post in its
>> entiretly:
>>
>> --
>>
>> WARNING: Using incubator modules: jdk.incubator.httpclient
>> Exception in thread "main" java.io.EOFException: EOF reached while reading
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.Http1AsyncReceiver$Http1TubeSubscriber.onComplete(Http1AsyncReceiver.java:507)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadSubscription.signalCompletion(SocketTube.java:551)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:728)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowTask.run(SocketTube.java:171)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:675)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:829)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:243)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:769)
>> at
>> jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:731)
>>
>> Process finished with exit code 1
>> --
>>
>> If I use a curl to make the request of my (node/express) server, I get:
>>
>> --
>> $ curl -v http://localhost:8080/index.html
>> *   Trying 127.0.0.1...
>> * Connected to localhost (127.0.0.1) port 8080 (#0)
>> > GET /index.html HTTP/1.1
>> > Host: localhost:8080
>> > User-Agent: curl/7.47.0
>> > Accept: */*
>> >
>> < HTTP/1.1 200 OK
>> < X-Powered-By: Express
>> < Content-Type: text/html; charset=utf-8
>> < Content-Length: 58
>> < ETag: W/"3a-EwoPOQKsJivlqZA3z/ulngzMv9U"
>> < Date: Tue, 15 May 2018 00:18:47 GMT
>> < Connection: keep-alive
>> <
>> * Connection #0 to host localhost left intact
>> HeadingSome Text
>> --
>>
>> Anyone able to tell me what I'm doing wrong?
>>
>> TIA!
>>
>>
>>
>

-- 
Simon Roberts
(303) 249 3613

Re:

[Joakim Erdfelt]

You're server is reporting the wrong Content-Length.

"HeadingSome Text" is 59
characters

But the server said "Content-Length: 58"

Also, "Connection: keep-alive" is a HTTP/1.0 concept only, but is being
reported on your HTTP/1.1 request/response improperly.


On Mon, May 14, 2018 at 7:20 PM, Simon Roberts <
si...@dancingcloudservices.com> wrote:

>
> I have a very simple attempt to use the Java 10 / incubator httpclient
> api. It works on "regular" resources, but fails horribly when attempting to
> get things from localhost. It fails using 127.0.0.1, the actual
> non-loopback IP, and the name localhost. The server is running, and
> presenting a very simple web page (I originally tried json, but have since
> simplified it attempting to find out what's amiss).
>
> The code is as simple as I can imagine (and that might be the problem :)
>
> --
> HttpClient client = HttpClient.newHttpClient();
> URI uri = URI.create("http://192.168.1.102:8080/index.html;);
>
> HttpRequest getRequest = HttpRequest.newBuilder()
> .uri(uri).GET().build();
>
> HttpResponse response = client.send(getRequest,
> HttpResponse.BodyHandler.asString());
> System.out.println("response to get: " + response.body());
> --
>
> The error prints a long stack trace, which I suppose I must post in its
> entiretly:
>
> --
>
> WARNING: Using incubator modules: jdk.incubator.httpclient
> Exception in thread "main" java.io.EOFException: EOF reached while reading
> at jdk.incubator.httpclient/jdk.incubator.http.Http1AsyncReceiver$
> Http1TubeSubscriber.onComplete(Http1AsyncReceiver.java:507)
> at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$
> InternalReadPublisher$ReadSubscription.signalCompletion(SocketTube.
> java:551)
> at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$
> InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:728)
> at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$
> SocketFlowTask.run(SocketTube.java:171)
> at jdk.incubator.httpclient/jdk.incubator.http.internal.
> common.SequentialScheduler$SchedulableTask.run(
> SequentialScheduler.java:198)
> at jdk.incubator.httpclient/jdk.incubator.http.internal.
> common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
> at jdk.incubator.httpclient/jdk.incubator.http.internal.
> common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
> at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$
> InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.
> java:675)
> at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$
> InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:829)
> at jdk.incubator.httpclient/jdk.incubator.http.SocketTube$
> SocketFlowEvent.handle(SocketTube.java:243)
> at jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$
> SelectorManager.handleEvent(HttpClientImpl.java:769)
> at jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$
> SelectorManager.run(HttpClientImpl.java:731)
>
> Process finished with exit code 1
> --
>
> If I use a curl to make the request of my (node/express) server, I get:
>
> --
> $ curl -v http://localhost:8080/index.html
> *   Trying 127.0.0.1...
> * Connected to localhost (127.0.0.1) port 8080 (#0)
> > GET /index.html HTTP/1.1
> > Host: localhost:8080
> > User-Agent: curl/7.47.0
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < X-Powered-By: Express
> < Content-Type: text/html; charset=utf-8
> < Content-Length: 58
> < ETag: W/"3a-EwoPOQKsJivlqZA3z/ulngzMv9U"
> < Date: Tue, 15 May 2018 00:18:47 GMT
> < Connection: keep-alive
> <
> * Connection #0 to host localhost left intact
> HeadingSome Text
> --
>
> Anyone able to tell me what I'm doing wrong?
>
> TIA!
>
>
>

Re:

[Daniel Fuchs]

Hi Simon,

We are using localhost/127.0.0.1 in all our tests so I doubt this
is the problem here. It's difficult to say what is happening with
just the stack trace.
If you're on Mac that could be an issue with the system proxy
configuration.

Could you please log a bug through
https://bugreport.java.com/bugreport/
and include a reproducer?

best regards,

-- daniel

On 15/05/2018 01:20, Simon Roberts wrote:


I have a very simple attempt to use the Java 10 / incubator httpclient 
api. It works on "regular" resources, but fails horribly when attempting 
to get things from localhost. It fails using 127.0.0.1, the actual 
non-loopback IP, and the name localhost. The server is running, and 
presenting a very simple web page (I originally tried json, but have 
since simplified it attempting to find out what's amiss).


The code is as simple as I can imagine (and that might be the problem :)

--
     HttpClient client = HttpClient.newHttpClient();
     URI uri = URI.create("http://192.168.1.102:8080/index.html;);

     HttpRequest getRequest = HttpRequest.newBuilder()
         .uri(uri).GET().build();

     HttpResponse response = client.send(getRequest,
         HttpResponse.BodyHandler.asString());
     System.out.println("response to get: " + response.body());
--

The error prints a long stack trace, which I suppose I must post in its 
entiretly:


--

WARNING: Using incubator modules: jdk.incubator.httpclient
Exception in thread "main" java.io.EOFException: EOF reached while reading
at 
jdk.incubator.httpclient/jdk.incubator.http.Http1AsyncReceiver$Http1TubeSubscriber.onComplete(Http1AsyncReceiver.java:507)
at 
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadSubscription.signalCompletion(SocketTube.java:551)
at 
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(SocketTube.java:728)
at 
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowTask.run(SocketTube.java:171)
at 
jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
at 
jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
at 
jdk.incubator.httpclient/jdk.incubator.http.internal.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
at 
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(SocketTube.java:675)
at 
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(SocketTube.java:829)
at 
jdk.incubator.httpclient/jdk.incubator.http.SocketTube$SocketFlowEvent.handle(SocketTube.java:243)
at 
jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.handleEvent(HttpClientImpl.java:769)
at 
jdk.incubator.httpclient/jdk.incubator.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:731)


Process finished with exit code 1
--

If I use a curl to make the request of my (node/express) server, I get:

--
$ curl -v http://localhost:8080/index.html
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
 > GET /index.html HTTP/1.1
 > Host: localhost:8080
 > User-Agent: curl/7.47.0
 > Accept: */*
 >
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Content-Type: text/html; charset=utf-8
< Content-Length: 58
< ETag: W/"3a-EwoPOQKsJivlqZA3z/ulngzMv9U"
< Date: Tue, 15 May 2018 00:18:47 GMT
< Connection: keep-alive
<
* Connection #0 to host localhost left intact
HeadingSome Text
--

Anyone able to tell me what I'm doing wrong?

TIA!

Re: RFR:8194298 Add support for per Socket configuration of TCP keepalive

[vyom tewari]

On Tuesday 15 May 2018 03:13 PM, Alan Bateman wrote:

On 15/05/2018 08:35, Langer, Christoph wrote:


I’m asking because I’m planning to add some AIX options and will have 
to choose a name for this implementation eventually.


@Alan: What do you think?


Yes, I agree it should be renamed. Vyom has just finalized the CSR so 
I assume the final points around the implementation and naming of the 
JDK internal classes can be sorted out while the CSR is in progress.


sure, i will rename the files to 
MacOSXSocketOptions.java I will send the modified 
webrev soon. Please do let me know if any buddy have better name 
suggestion than this.


Thanks,
vyom

-Alan.

Re: RFR:8194298 Add support for per Socket configuration of TCP keepalive

[Alan Bateman]

On 15/05/2018 08:35, Langer, Christoph wrote:


I’m asking because I’m planning to add some AIX options and will have 
to choose a name for this implementation eventually.


@Alan: What do you think?


Yes, I agree it should be renamed. Vyom has just finalized the CSR so I 
assume the final points around the implementation and naming of the JDK 
internal classes can be sorted out while the CSR is in progress.


-Alan.

RE: RFR:8194298 Add support for per Socket configuration of TCP keepalive

[Langer, Christoph]

Hi,

As I had written previously, I’m still unclear whether the naming of the MacOS 
implementation files is the best choice.

Currently it is:
src/jdk.net/macosx/classes/jdk/net/UnixSocketOptions.java
src/jdk.net/macosx/native/libextnet/UnixSocketOptions.c

Maybe it  should rather be:
src/jdk.net/macosx/classes/jdk/net/MacOSXSocketOptions.java
src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c

Since for Linux we have:
src/jdk.net/linux/classes/jdk/net/LinuxSocketOptions.java
src/jdk.net/linux/native/libextnet/LinuxSocketOptions.c

I’m asking because I’m planning to add some AIX options and will have to choose 
a name for this implementation eventually.

@Alan: What do you think?

Best regards
Christoph


From: net-dev [mailto:net-dev-boun...@openjdk.java.net] On Behalf Of vyom tewari
Sent: Montag, 14. Mai 2018 17:31
To: Alan Bateman ; OpenJDK Network Dev list 

Subject: Re: RFR:8194298 Add support for per Socket configuration of TCP 
keepalive


Hi All,

Please find the latest 
webrev(http://cr.openjdk.java.net/~vtewari/8194298/webrev0.5/index.html). Only 
change with the previous wrev(04) is i removed "socket type" as Alan suggested 
and used the default  constructor (Set options = new 
HashSet<>();) in ExtendedSocketOptions.java

Thanks,

Vyom

On Sunday 13 May 2018 12:37 PM, Alan Bateman wrote:
On 12/05/2018 10:21, vyom tewari wrote:

:

Thanks for review, please find the updated 
webrev(http://cr.openjdk.java.net/~vtewari/8194298/webrev0.4/index.html).
I've skimmed through this webrev.

The spec for the new options mostly look good but all three include "The exact 
semantics of this socket option are socket type and system dependent". I assume 
"socket type" should be dropped from this as these are TCP options. Maybe you 
can borrow text from the TCP_NODELAY option where it has the statement "The 
socket option is specific to stream-oriented sockets using the TCP/IP protocol".

The changes to the NetworkChannel implementations look okay. The changes to the 
NIO tests looks okay too but would be good if you could keep the formatting 
consistent with the existing code if you can.

Ivan had good comments so I didn't spent as much time on that code and it seems 
very reviewed.

-Alan.