Hi Simon, Thanks, that's a very likely explanation already. I just added the pbuf_free(p) but run into an assert occasionally.
The pbuf is already freed, but that happens always after a bad fcs. The bad fcs is because I miss a byte from the received ppp packet. So I will fix that problem first, and then will see where I end up. If it still fails to download I will share the pcap files and hope you can have a look. Thanks Best regards Bas Op wo 9 jun. 2021 om 13:42 schreef [email protected] <[email protected]>: > Am 09.06.2021 um 12:42 schrieb Bas Prins: > > Hi > > > > I am having a hard time understanding the http_client.h interface. > > > > My setup: > > - freertos latest version > > - lwip 2.1.2 (over pppos) > > - the only tasks running are lwip related (tcpip thread (task prio=2 + > > rx thread (task prio=2). > > - both are over dimensioned for stack and heap (because I'm still > > developing, I'll reduce to decent amounts when I need to) > > - uart2 rx/tx are connected to LTE modem. Both rx and tx is used in > > interrupt mode. > > - uart3 tx is used for logging to PC (where some simple C# app reads all > > data and logs to file) > > > > I am connecting to a web-server where I want to download a "512.KB.zip" > > (zip contains \0 characters, I took it from some download speed test > > server). > > > > I experimented with lwftp <https://github.com/gezedo/lwftp>, but it > > I don't know that one. > > > seems to leak memory the way I use it. So I thought, why not just > > download files using http get and use the lwip provided http_client. > > > > The following code is used to initiate the download : > > > > LogLineToUart("connecting to server...\n"); > > > > ip_addr_t ip; > > uint16_t port= 80; > > httpc_connection_t settings; > > > > IP4_ADDR(&ip, 31,21,236,6); > > settings.result_fn = result_fn; > > > > err_t err = httpc_get_file( > > &ip, > > port, > > "/512KB.zip", > > &settings, > > rec_fn, > > nullptr, > > nullptr); > > > > if ( err != ERR_OK ) > > { > > LogLineToUart("httpc_get_file failed (%d)", err); > > } > > > > for now, I only log when I am receiving data: > > > > > err_t rec_fn(void *arg, struct altcp_pcb *conn, struct pbuf *p, err_t err) > > { > > LogLineToUart("sinking: %d\n", p->len); > > You should be freeing the pbuf here or you'll run out of pbufs eventually. > > > } > > > > > void result_fn(void *arg, httpc_result_t httpc_result, u32_t rx_content_len, > u32_t srv_res, err_t err) > > { > > LogLineToUart("download finished\n"); > > } > > > > Both functions are called. So data is being received. > > > > The questions arise when I look at wireshark. > > > > 1278302.433400192.168.1.113185.99.25.76TCP5880 → 62510 [SYN, ACK] Seq=0 > > Ack=1 Win=29200 Len=0 MSS=1460 > > 1282302.871908185.99.25.76192.168.1.113HTTP205GET /ftp_100kb_5.log > HTTP/1.1 > > This is not requesting "/512KB.zip" as in the code above? > > > 1283302.872167192.168.1.113185.99.25.76TCP5480 → 62510 [ACK] Seq=1 > > Ack=152 Win=29736 Len=0 > > 1284302.873849192.168.1.113185.99.25.76TCP52680 → 62510 [ACK] Seq=1 > > Ack=152 Win=29736 Len=472 [TCP segment of a reassembled PDU] > > 1285302.874100192.168.1.113185.99.25.76TCP52680 → 62510 [ACK] Seq=473 > > Ack=152 Win=29736 Len=472 [TCP segment of a reassembled PDU] > > 1286302.874253192.168.1.113185.99.25.76TCP52680 → 62510 [ACK] Seq=945 > > Ack=152 Win=29736 Len=472 [TCP segment of a reassembled PDU] > > 1287302.874394192.168.1.113185.99.25.76TCP52680 → 62510 [ACK] Seq=1417 > > Ack=152 Win=29736 Len=472 [TCP segment of a reassembled PDU] > > 1288302.874535192.168.1.113185.99.25.76TCP52680 → 62510 [ACK] Seq=1889 > > Ack=152 Win=29736 Len=472 [TCP segment of a reassembled PDU] > > 1289302.874688192.168.1.113185.99.25.76TCP52680 → 62510 [PSH, ACK] > > Seq=2361 Ack=152 Win=29736 Len=472 [TCP segment of a reassembled PDU] > > 1290302.874835192.168.1.113185.99.25.76TCP52680 → 62510 [ACK] Seq=2833 > > Ack=152 Win=29736 Len=472 [TCP segment of a reassembled PDU] > > 1291302.874975192.168.1.113185.99.25.76TCP52680 → 62510 [ACK] Seq=3305 > > Ack=152 Win=29736 Len=472 [TCP segment of a reassembled PDU] > > 1292302.875118192.168.1.113185.99.25.76TCP52680 → 62510 [ACK] Seq=3777 > > Ack=152 Win=29736 Len=472 [TCP segment of a reassembled PDU] > > 1293302.875261192.168.1.113185.99.25.76TCP526[TCP Window Full] 80 → > > 62510 [ACK] Seq=4249 Ack=152 Win=29736 Len=472 [TCP segment of a > > reassembled PDU] > > 1295303.465207185.99.25.76192.168.1.113TCP6062510 → 80 [ACK] Seq=152 > > Ack=473 Win=4248 Len=0 > > 1304304.795607192.168.1.113185.99.25.76TCP526[TCP Retransmission] 80 → > > 62510 [ACK] Seq=473 Ack=152 Win=29736 Len=472 > > 1305305.354254185.99.25.76192.168.1.113TCP60[TCP Dup ACK 1295#1] 62510 → > > 80 [ACK] Seq=152 Ack=473 Win=4248 Len=0 > > 1311307.485583192.168.1.113185.99.25.76TCP526[TCP Retransmission] 80 → > > 62510 [ACK] Seq=473 Ack=152 Win=29736 Len=472 > > 1312307.903785185.99.25.76192.168.1.113TCP60[TCP Dup ACK 1295#2] 62510 → > > 80 [ACK] Seq=152 Ack=473 Win=4248 Len=0 > > 1324312.855604192.168.1.113185.99.25.76TCP526[TCP Retransmission] 80 → > > 62510 [ACK] Seq=473 Ack=152 Win=29736 Len=472 > > 1326313.354377185.99.25.76192.168.1.113TCP60[TCP Dup ACK 1295#3] 62510 → > > 80 [ACK] Seq=152 Ack=473 Win=4248 Len=0 > > 1334317.187014185.99.25.76192.168.1.113TCP6062510 → 80 [RST, ACK] > > Seq=152 Ack=473 Win=28690 Len=0 > > > > The data sent by the server doesn't add up to 512KB by a long shot. And > > my STM doesn't seem to send acks fast enough, resulting in window full + > > retransmits. > > Please share pcap files, not text summaries. You can see in the pcap > which data is being sent by the server. That should help you to see what > might be wrong. > > Regards, > Simon > > > > > I also enabled lwip debugging, just to be 100% no packets are being > > dropped. They are not. > > > > I have been playing with lwipopt settings (window size, MSS, #memp,. > > etc) but nothing results in a better performance. > > > > Does somebody recognize these problems from when you just started > > working with lwip? I am running out of ideas on how to approach this. > > > > For what it's worth, my lwftp implementation works for the downloading > > part. The traffic in wireshark is good. Data is being sent, and lwip is > > responding with acks. Occasionally a retransmit but that's it. I also > > did a quick test with mqtt. Same story, works flawless, sending > > publishes every half second and capable of logging published messages > > from broker on the same rate. > > > > I suspect everything points to wrong lwipopts configuration. I'll post > > my config below, if there are any alarm bells ringing I would be very > > grateful if somebody would point them out to me. > > > > lwipopts.h > > > > /* > > * Copyright (c) 2001-2003 Swedish Institute of Computer Science. > > * All rights reserved. > > * > > > * Redistribution and use in source and binary forms, with or without > modification, > > * are permitted provided that the following conditions are met: > > * > > > * 1. Redistributions of source code must retain the above copyright notice, > > * this list of conditions and the following disclaimer. > > > * 2. Redistributions in binary form must reproduce the above copyright > notice, > > > * this list of conditions and the following disclaimer in the > documentation > > * and/or other materials provided with the distribution. > > > * 3. The name of the author may not be used to endorse or promote products > > > * derived from this software without specific prior written permission. > > * > > > * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR > IMPLIED > > * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF > > > * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO > EVENT > > > * SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, > > > * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, > PROCUREMENT > > > * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS > > > * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN > > > * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) > ARISING > > > * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE > POSSIBILITY > > * OF SUCH DAMAGE. > > * > > * This file is part of the lwIP TCP/IP stack. > > * > > * Author: Adam Dunkels <[email protected] <mailto:[email protected]>> > > * > > */ > > #ifndef LWIP_LWIPOPTS_H > > #define LWIP_LWIPOPTS_H > > > > #ifdef LWIP_OPTTEST_FILE > > #else /* LWIP_OPTTEST_FILE */ > > > > #define LWIP_IPV4 1 > > #define LWIP_IPV6 0 > > > > #define NO_SYS 0 > > #define LWIP_SOCKET (NO_SYS==0) > > #define LWIP_NETCONN (NO_SYS==0) > > #define LWIP_NETIF_API (NO_SYS==0) > > > > #define LWIP_IGMP 0 > > #define LWIP_ICMP 0 > > > > #ifdef LWIP_UDP > > #define LWIP_SNMP LWIP_UDP > > #define MIB2_STATS LWIP_SNMP > > > > #define LWIP_DNS LWIP_UDP > > #define LWIP_MDNS_RESPONDER LWIP_UDP > > > > #define LWIP_NUM_NETIF_CLIENT_DATA (LWIP_MDNS_RESPONDER) > > #endif > > > > > > #define LWIP_HAVE_LOOPIF 0 > > #define LWIP_NETIF_LOOPBACK 0 > > #define LWIP_LOOPBACK_MAX_PBUFS 0 > > > > #define TCP_LISTEN_BACKLOG 1 > > > > #define TCPIP_MBOX_SIZE 10 > > #define TCPIP_THREAD_STACKSIZE 4000 > > #define DEFAULT_THREAD_STACKSIZE 3000 > > #define DEFAULT_TCP_RECVMBOX_SIZE 10 > > > > #define LWIP_COMPAT_SOCKETS 1 > > #define LWIP_SO_RCVTIMEO 1 > > #define LWIP_SO_RCVBUF 1 > > > > #define LWIP_FREERTOS_CHECK_CORE_LOCKING 1 > > #define LWIP_TCPIP_CORE_LOCKING 1 > > > > #define LWIP_NETIF_LINK_CALLBACK 1 > > #define LWIP_NETIF_STATUS_CALLBACK 1 > > #define LWIP_NETIF_EXT_STATUS_CALLBACK 1 > > > > #define TCPIP_THREAD_PRIO 2 > > > > // Uncomment next line if more lwip logging is needed > > #define LWIP_DEBUG > > > > #ifdef LWIP_DEBUG > > > > #define LWIP_DBG_MIN_LEVEL 0 > > #define PPP_DEBUG LWIP_DBG_ON > > #define MEM_DEBUG LWIP_DBG_OFF > > #define MEMP_DEBUG LWIP_DBG_OFF > > #define PBUF_DEBUG LWIP_DBG_OFF > > #define API_LIB_DEBUG LWIP_DBG_OFF > > #define API_MSG_DEBUG LWIP_DBG_OFF > > #define TCPIP_DEBUG LWIP_DBG_OFF > > #define NETIF_DEBUG LWIP_DBG_OFF > > #define SOCKETS_DEBUG LWIP_DBG_OFF > > #define DNS_DEBUG LWIP_DBG_OFF > > #define AUTOIP_DEBUG LWIP_DBG_OFF > > #define DHCP_DEBUG LWIP_DBG_OFF > > #define IP_DEBUG LWIP_DBG_OFF > > #define IP_REASS_DEBUG LWIP_DBG_OFF > > #define ICMP_DEBUG LWIP_DBG_OFF > > #define IGMP_DEBUG LWIP_DBG_OFF > > #define UDP_DEBUG LWIP_DBG_OFF > > #define TCP_DEBUG LWIP_DBG_OFF > > #define TCP_INPUT_DEBUG LWIP_DBG_OFF > > #define TCP_OUTPUT_DEBUG LWIP_DBG_OFF > > #define TCP_RTO_DEBUG LWIP_DBG_OFF > > #define TCP_CWND_DEBUG LWIP_DBG_OFF > > #define TCP_WND_DEBUG LWIP_DBG_OFF > > #define TCP_FR_DEBUG LWIP_DBG_OFF > > #define TCP_QLEN_DEBUG LWIP_DBG_OFF > > #define TCP_RST_DEBUG LWIP_DBG_OFF > > #endif > > > > > #define LWIP_DBG_TYPES_ON > (LWIP_DBG_ON|LWIP_DBG_TRACE|LWIP_DBG_STATE|LWIP_DBG_FRESH|LWIP_DBG_HALT) > > > > > > /* ---------- Memory options ---------- */ > > /* MEM_ALIGNMENT: should be set to the alignment of the CPU for which > > lwIP is compiled. 4 byte alignment -> define MEM_ALIGNMENT to 4, 2 > > byte alignment -> define MEM_ALIGNMENT to 2. */ > > /* MSVC port: intel processors don't need 4-byte alignment, > > but are faster that way! */ > > #define MEM_ALIGNMENT 4U > > > > #define MEMP_SANITY_CHECK 1 > > > > /** > > * MEMP_OVERFLOW_CHECK: memp overflow protection reserves a configurable > > > * amount of bytes before and after each memp element in every pool and fills > > * it with a prominent default value. MEMP_OVERFLOW_CHECK == 0 no > > * checking MEMP_OVERFLOW_CHECK == 1 checks each element when it is > > * freed MEMP_OVERFLOW_CHECK >= 2 checks each element in every pool > > * every time memp_malloc() or memp_free() is called (useful but slow!) > > */ > > #define MEMP_OVERFLOW_CHECK 1 > > > > /* MEM_SIZE: the size of the heap memory. If the application will send > > a lot of data that needs to be copied, this should be set high. */ > > #define MEM_SIZE (30*1024) > > > > /* MEMP_NUM_PBUF: the number of memp struct pbufs. If the application > > sends a lot of data out of ROM (or other static memory), this > > should be set high. */ > > #define MEMP_NUM_PBUF 100 > > /* MEMP_NUM_RAW_PCB: the number of UDP protocol control blocks. One > > per active RAW "connection". */ > > #define MEMP_NUM_RAW_PCB 0 > > /* MEMP_NUM_UDP_PCB: the number of UDP protocol control blocks. One > > per active UDP "connection". */ > > #define MEMP_NUM_UDP_PCB 0 > > /* MEMP_NUM_TCP_PCB: the number of simulatenously active TCP > > connections. */ > > #define MEMP_NUM_TCP_PCB 6 > > /* MEMP_NUM_TCP_PCB_LISTEN: the number of listening TCP > > connections. */ > > #define MEMP_NUM_TCP_PCB_LISTEN 0 > > /* MEMP_NUM_TCP_SEG: the number of simultaneously queued TCP > > segments. */ > > #define MEMP_NUM_TCP_SEG 16 > > /* MEMP_NUM_SYS_TIMEOUT: the number of simulateously active > > timeouts. */ > > #define MEMP_NUM_SYS_TIMEOUT 17 > > > > /* The following four are used only with the sequential API and can be > > set to 0 if the application only will use the raw API. */ > > /* MEMP_NUM_NETBUF: the number of struct netbufs. */ > > #define MEMP_NUM_NETBUF 8 > > /* MEMP_NUM_NETCONN: the number of struct netconns. */ > > #define MEMP_NUM_NETCONN 12 > > /* MEMP_NUM_TCPIP_MSG_*: the number of struct tcpip_msg, which is used > > for sequential API communication and incoming packets. Used in > > src/api/tcpip.c. */ > > #define MEMP_NUM_TCPIP_MSG_API 16 > > #define MEMP_NUM_TCPIP_MSG_INPKT 16 > > > > > > /** SYS_LIGHTWEIGHT_PROT > > > * define SYS_LIGHTWEIGHT_PROT in lwipopts.h if you want inter-task protection > > > * for certain critical regions during buffer allocation, deallocation and > memory > > * allocation and deallocation. > > */ > > #define SYS_LIGHTWEIGHT_PROT (NO_SYS==0) > > > > /** > > * TODO BP > > * Check opt.h for reference > > * This might be a bad idea > > */ > > #define ETH_PAD_SIZE 2 > > > > > > /* ---------- TCP options ---------- */ > > #define LWIP_TCP 1 > > #define TCP_TTL 255 > > > > #define LWIP_ALTCP (LWIP_TCP) > > #ifdef LWIP_HAVE_MBEDTLS > > #define LWIP_ALTCP_TLS (LWIP_TCP) > > #define LWIP_ALTCP_TLS_MBEDTLS (LWIP_TCP) > > #endif > > > > > > /* Controls if TCP should queue segments that arrive out of > > order. Define to 0 if your device is low on memory. */ > > #define TCP_QUEUE_OOSEQ 1 > > > > /* TCP Maximum segment size. */ > > > #define TCP_MSS (512 - 40) /* TCP_MSS = (Ethernet MTU - > IP header size - TCP header size) */ > > > > /* TCP receive window. */ > > #define TCP_WND (10*TCP_MSS) > > > > /* TCP sender buffer space (bytes). */ > > #define TCP_SND_BUF (2*TCP_MSS) > > > > /* PBUF_POOL_SIZE: the number of buffers in the pbuf pool. */ > > #define PBUF_POOL_SIZE 20 > > > > /* PBUF_POOL_BUFSIZE: the size of each pbuf in the pbuf pool. */ > > #define PBUF_POOL_BUFSIZE (TCP_MSS + 40 + 40) > > > > /* TCP sender buffer space (pbufs). This must be at least = 2 * > > TCP_SND_BUF/TCP_MSS for things to work. */ > > #define TCP_SND_QUEUELEN (4* TCP_SND_BUF/TCP_MSS) > > > > /* TCP writable space (bytes). This must be less than or equal > > to TCP_SND_BUF. It is the amount of space which must be > > available in the tcp snd_buf for select to return writable */ > > #define TCP_SNDLOWAT (TCP_SND_BUF/2) > > > > /* Maximum number of retransmissions of data segments. */ > > #define TCP_MAXRTX 12 > > > > /* Maximum number of retransmissions of SYN segments. */ > > #define TCP_SYNMAXRTX 4 > > > > /* ---------- ARP options ---------- */ > > #define LWIP_ARP 1 > > #define ARP_TABLE_SIZE 10 > > #define ARP_QUEUEING 1 > > > > > > /* ---------- IP options ---------- */ > > /* Define IP_FORWARD to 1 if you wish to have the ability to forward > > IP packets across network interfaces. If you are going to run lwIP > > on a device with only one network interface, define this to 0. */ > > #define IP_FORWARD 0 > > > > /* IP reassembly and segmentation.These are orthogonal even > > * if they both deal with IP fragments */ > > #define IP_REASSEMBLY 1 > > > #define IP_REASS_MAX_PBUFS (10 * ((1500 + PBUF_POOL_BUFSIZE - 1) / > PBUF_POOL_BUFSIZE)) > > #define MEMP_NUM_REASSDATA IP_REASS_MAX_PBUFS > > #define IP_FRAG 1 > > #define IPV6_FRAG_COPYHEADER 1 > > > > /* ---------- ICMP options ---------- */ > > #define ICMP_TTL 255 > > > > > > /* ---------- DHCP options ---------- */ > > /* Define LWIP_DHCP to 1 if you want DHCP configuration of > > interfaces. */ > > #define LWIP_DHCP LWIP_UDP > > > > /* 1 if you want to do an ARP check on the offered address > > (recommended). */ > > #define DHCP_DOES_ARP_CHECK (LWIP_DHCP) > > > > > > /* ---------- AUTOIP options ------- */ > > #define LWIP_AUTOIP (LWIP_DHCP) > > #define LWIP_DHCP_AUTOIP_COOP (LWIP_DHCP && LWIP_AUTOIP) > > > > > > /* ---------- UDP options ---------- */ > > #define LWIP_UDP 0 > > #define LWIP_UDPLITE LWIP_UDP > > #define UDP_TTL 255 > > > > > > /* ---------- RAW options ---------- */ > > #define LWIP_RAW 0 > > > > /* ---------- Statistics options ---------- */ > > > > #define LWIP_STATS 1 > > #define LWIP_STATS_DISPLAY 1 > > > > #if LWIP_STATS > > #define LINK_STATS 0 > > #define IP_STATS 0 > > #define ICMP_STATS 0 > > #define IGMP_STATS 0 > > #define IPFRAG_STATS 0 > > #define UDP_STATS 0 > > #define TCP_STATS 0 > > #define MEM_STATS 1 > > #define MEMP_STATS 1 > > #define PBUF_STATS 0 > > #define SYS_STATS 1 > > #endif /* LWIP_STATS */ > > > > /* ---------- NETBIOS options ---------- */ > > #define LWIP_NETBIOS_RESPOND_NAME_QUERY 0 > > > > /* ---------- PPP options ---------- */ > > > > #define PPP_SUPPORT 1 /* Set > 0 for PPP */ > > > > #if PPP_SUPPORT > > > > #define PRINTPKT_SUPPORT 0 // TODO BP, added for debugging > > #define NUM_PPP 1 /* Max PPP sessions. */ > > #define PPP_NOTIFY_PHASE 1 // TODO BP, added for debugging > > > > > /* Select modules to enable. Ideally these would be set in the makefile but > > > * we're limited by the command line length so you need to modify the settings > > * in this file. > > */ > > #define PPPOE_SUPPORT 0 > > #define PPPOS_SUPPORT 1 > > > > #define PAP_SUPPORT 0 /* Set > 0 for PAP. */ > > #define CHAP_SUPPORT 0 /* Set > 0 for CHAP. */ > > #define MSCHAP_SUPPORT 0 /* Set > 0 for MSCHAP */ > > > #define CBCP_SUPPORT 0 /* Set > 0 for CBCP (NOT FUNCTIONAL!) > */ > > #define CCP_SUPPORT 0 /* Set > 0 for CCP */ > > > #define VJ_SUPPORT 0 /* Set > 0 for VJ header compression. > */ > > > #define MD5_SUPPORT 0 /* Set > 0 for MD5 (see also CHAP) */ > > > > #endif /* PPP_SUPPORT */ > > > > #endif /* LWIP_OPTTEST_FILE */ > > > > /* The following defines must be done even in OPTTEST mode: */ > > > > #if !defined(NO_SYS) || !NO_SYS /* default is 0 */ > > void sys_check_core_locking(void); > > #define LWIP_ASSERT_CORE_LOCKED() sys_check_core_locking() > > #endif > > > > #endif /* LWIP_LWIPOPTS_H */ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > lwip-users mailing list > > [email protected] > > https://lists.nongnu.org/mailman/listinfo/lwip-users > > > > _______________________________________________ > lwip-users mailing list > [email protected] > https://lists.nongnu.org/mailman/listinfo/lwip-users
_______________________________________________ lwip-users mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/lwip-users
