Re: [lwip-users] Slow HTTP put request
> BTW: curl wants "HTTP/1.0 100 Continue\r\n\r\n" in the reply. > "HTTP/1.0 100 Continue\r\n\" leads to confusion. I guess you mean curl wants "HTTP/1.1 100 Continue\r\n\r\n"; and that is because 100 Continue belongs to the 1.1 spec, there is no such thing on the HTTP specification 1.0 ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
Speaking of that, why do you use PUT anyway? I would have used POST to implement what you seem to do. PUT seems rather "unusual" to me... I thought PUT - because of its history? - is the right thing to send binary data to a server e.g. for firmware update. And a PUT header seems to be easier to create and decode than a POST multipart header. But yes, regarding the article in the previous post labview PUT seems to be unusual and POST seems to be better supported on labview. I'll try to check this out. I don't know how that client knows about the version of the remote server. It can issue a first request to get the server's version, but that seems rather unusual, too. And from your traces, that doesn't happen... But you can try to just make the server report "HTTP/1.0" in every response instead of "HTTP/1.1" and see what happens... Regarding the PUT problem labview obviously doesn't care about my HTTP 1.0 replies and still requests expect/continue handing from the server otherwise the server is punished with a second delay. Maybe there is a labview configuration option for HTTP 1.0 ? BTW: curl wants "HTTP/1.0 100 Continue\r\n\r\n" in the reply. "HTTP/1.0 100 Continue\r\n\" leads to confusion. ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
joc...@strohbeck.net wrote: > [..] > Regarding the expect/continue overhead, according to wireshark trace > below it is about 30ms extra - way to much for what I intended to use > PUT (write a couple of bytes to twi interface). Speaking of that, why do you use PUT anyway? I would have used POST to implement what you seem to do. PUT seems rather "unusual" to me... > [..] > I always thought there is a mechanism to tell the client that my server > supports HTTP 1.0 only and this should prevent the client from asking me > things I do no support. Am I wrong ? I don't know how that client knows about the version of the remote server. It can issue a first request to get the server's version, but that seems rather unusual, too. And from your traces, that doesn't happen... But you can try to just make the server report "HTTP/1.0" in every response instead of "HTTP/1.1" and see what happens... Simon ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
I see the following solution: - server sends reply to expect/continue and then... well, frankly don't know exactly what happens then and how much overhead this means [..] I see 2 solutions: implement a HTTP 1.1 server that keeps to the specification (I'll have to do that for the lwIP httpd, too) or implement a HTTP 1.0 server that keeps to the 1.0 specification (in which case the client should not depend on the Expect header, I guess). Regarding the expect/continue overhead, according to wireshark trace below it is about 30ms extra - way to much for what I intended to use PUT (write a couple of bytes to twi interface). I don't know what it means to me to implement a full fashioned HTTP 1.1 compatible server. There are http implementations running on systems with a couple of bytes of ram and rom. I this means extra code, CPU time and more testing I'd like to keep to HTTP 1.0. I always thought there is a mechanism to tell the client that my server supports HTTP 1.0 only and this should prevent the client from asking me things I do no support. Am I wrong ? curl_put_continue.pcapng Description: Binary data ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
joc...@strohbeck.net wrote: > I see the following solution: > - server sends reply to expect/continue and then... well, frankly don't > know exactly what happens then and how much overhead this means > [..] I see 2 solutions: implement a HTTP 1.1 server that keeps to the specification (I'll have to do that for the lwIP httpd, too) or implement a HTTP 1.0 server that keeps to the 1.0 specification (in which case the client should not depend on the Expect header, I guess). Simon ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
You should capture the traffic on your customer premises to make sure this is what's happening. I'm curious to know. Below is the trace of our customer. I think if you look at packets around no 2552 for example it shows the same behavior as curl in my understanding. Labview is waiting with a 1 sec timeout for the server reply to expect/continue which does not happen and then continues to send the 1 byte payload. This slows down the PUT dramatically. My server has also implemented POST and here the payload is received without any delay (packets around no 2540). It seems there is no additional "expect/continue handshake" implemented? I found an article (link below) regarding labview && POST and as I understand using POST this "handshake" is only done if the message is greater than 1024. For PUT there is no such limit ? I see the following solution: - server sends reply to expect/continue and then... well, frankly don't know exactly what happens then and how much overhead this means - force labview to "HTTP 1.0" w/o expect/continue ... if this is possible ? - implement an "application specific protocol" in labview as suggested in the article below ? - use POST instead of PUT for my "TWI write" function in order to speed up transfers below 1024 bytes ? Let me know if this thread is getting off-topic. Links: https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z019LuWSAU labview_post.pcapng Description: Binary data labview_put.pcapng Description: Binary data ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
> Try `curl -H "Expect:"` to remove it, or even `curl -0` to fallback to > HTTP1.0; but it won't help in your real life scenario. We don't know if > that is what happens on your customer side. Works like a charm! > You should capture the traffic on your customer premises to make sure > this is what's happening. I'm curious to know. I'll do that next week and let you know. Thanks a lot Sergio, thank you Simon ! ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
Sergio R. Caprile wrote: >You should capture the traffic on your customer premises to make sure >this is what's happening. I'm curious to know. You could also implement handling the Expect header in your http server and see if this speeds up the transfer. Simon ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
On 24.03.2018 07:26, joc...@strohbeck.net wrote: [..] Attached is the pcap file. All I can tell is that the header and payload is split into 2 packets and using curl and labview the 2nd packet is received after a huge delay. I tested --no-delay and --no-buffer and added manually keep-alive to the http header but no success. Hope you see something. OK, have you checked the difference between the 2 transfers? The 2nd (slower) transfer sends "Expect: 100-continue" in the HTTP headers. A HTTP 1.1 server conforming to the standard must either send a matching response or an error. I guess you don't have a server that keeps to this standard :-) (Although I think our httpd does not conform to this, either...) Simon ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
Am 23.03.2018 21:10, schrieb goldsi...@gmx.de: On 23.03.2018 20:50, Jochen Strohbeck wrote: [..] I am able to reproduce the same problem using curl on windows and found out that even a PUT request with only a single byte payload takes about a second! If I do the same request with python it takes some milliseconds. Why? I guess the problem is not directly related to lwip but I do not have any clue where the problem is located. I put the wireshark output below. First is the output of the curl request, second the same request using python. Any ideas ? Yes: send a pcap, not some text. From the text output you sent, it seems obvious that the 1st version delays packets. Without having a look at the packet details, it's hard to tell why. Simon Attached is the pcap file. All I can tell is that the header and payload is split into 2 packets and using curl and labview the 2nd packet is received after a huge delay. I tested --no-delay and --no-buffer and added manually keep-alive to the http header but no success. Hope you see something. requests_vs_curl.pcapng Description: Binary data ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] Slow HTTP put request
On 23.03.2018 20:50, Jochen Strohbeck wrote: [..] I am able to reproduce the same problem using curl on windows and found out that even a PUT request with only a single byte payload takes about a second! If I do the same request with python it takes some milliseconds. Why? I guess the problem is not directly related to lwip but I do not have any clue where the problem is located. I put the wireshark output below. First is the output of the curl request, second the same request using python. Any ideas ? Yes: send a pcap, not some text. From the text output you sent, it seems obvious that the 1st version delays packets. Without having a look at the packet details, it's hard to tell why. Simon ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
[lwip-users] Slow HTTP put request
Sorry for the dumb subject but I don't know better... I implemented a HTTP server using the netconn API on a SAME70 board and tested ethernet communication using python scripts. Now our customer did the same with labview on windows and the communication is slow especially using PUT requests. I am able to reproduce the same problem using curl on windows and found out that even a PUT request with only a single byte payload takes about a second! If I do the same request with python it takes some milliseconds. Why? I guess the problem is not directly related to lwip but I do not have any clue where the problem is located. I put the wireshark output below. First is the output of the curl request, second the same request using python. Any ideas ? Jochen 13 11.971296 192.168.0.200 192.168.0.100 TCP 66 24773 → 80 [SYN] Seq=0 Win=8192 Len=0 MSS=1260 WS=4 SACK_PERM=1 14 11.971481 192.168.0.100 192.168.0.200 TCP 60 80 → 24773 [SYN, ACK] Seq=0 Ack=1 Win=2920 Len=0 MSS=1460 15 11.971632 192.168.0.200 192.168.0.100 TCP 54 24773 → 80 [ACK] Seq=1 Ack=1 Win=65520 Len=0 16 11.978719 192.168.0.200 192.168.0.100 TCP 203 24773 → 80 [PSH, ACK] Seq=1 Ack=1 Win=65520 Len=149 [TCP segment of a reassembled PDU] 17 12.222453 192.168.0.100 192.168.0.200 TCP 60 80 → 24773 [ACK] Seq=1 Ack=150 Win=2771 Len=0 18 13.000423 192.168.0.200 192.168.0.100 HTTP 55 PUT /twi0/66 HTTP/1.1 19 13.001296 192.168.0.100 192.168.0.200 TCP 71 80 → 24773 [PSH, ACK] Seq=1 Ack=151 Win=2770 Len=17 [TCP segment of a reassembled PDU] 20 13.001480 192.168.0.100 192.168.0.200 HTTP 157 HTTP/1.0 200 OK (application/json) 21 13.001576 192.168.0.200 192.168.0.100 TCP 54 24773 → 80 [ACK] Seq=151 Ack=122 Win=65400 Len=0 22 13.008113 192.168.0.200 192.168.0.100 TCP 54 24773 → 80 [FIN, ACK] Seq=151 Ack=122 Win=65400 Len=0 23 13.008330 192.168.0.100 192.168.0.200 TCP 60 80 → 24773 [ACK] Seq=122 Ack=152 Win=2769 Len=0 9 6.245329 192.168.0.200 192.168.0.100 TCP 66 24461 → 80 [SYN] Seq=0 Win=8192 Len=0 MSS=1260 WS=4 SACK_PERM=1 10 6.245554 192.168.0.100 192.168.0.200 TCP 60 80 → 24461 [SYN, ACK] Seq=0 Ack=1 Win=2920 Len=0 MSS=1460 11 6.245667 192.168.0.200 192.168.0.100 TCP 54 24461 → 80 [ACK] Seq=1 Ack=1 Win=65520 Len=0 12 6.245913 192.168.0.200 192.168.0.100 TCP 224 24461 → 80 [PSH, ACK] Seq=1 Ack=1 Win=65520 Len=170 [TCP segment of a reassembled PDU] 13 6.246020 192.168.0.200 192.168.0.100 HTTP 55 PUT /twi0/66 HTTP/1.1 14 6.246243 192.168.0.100 192.168.0.200 TCP 60 80 → 24461 [ACK] Seq=1 Ack=172 Win=2749 Len=0 15 6.247102 192.168.0.100 192.168.0.200 TCP 71 80 → 24461 [PSH, ACK] Seq=1 Ack=172 Win=2749 Len=17 [TCP segment of a reassembled PDU] 16 6.247290 192.168.0.100 192.168.0.200 HTTP 157 HTTP/1.0 200 OK (application/json) 17 6.247368 192.168.0.200 192.168.0.100 TCP 54 24461 → 80 [ACK] Seq=172 Ack=122 Win=65400 Len=0 18 6.248011 192.168.0.200 192.168.0.100 TCP 54 24461 → 80 [FIN, ACK] Seq=172 Ack=122 Win=65400 Len=0 19 6.248210 192.168.0.100 192.168.0.200 TCP 60 80 → 24461 [ACK] Seq=122 Ack=173 Win=2748 Len=0 ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
