Hi Jerome, On 11/6/26 10:07, Jerome Forissier wrote: >> wget can write a disk image only after the whole file is in RAM. This >> series lets lwIP wget write the download to a block device while it >> arrives: >> >> => wget -o mmc 1 -b 400000 http://192.168.1.65/armbian.img >> >> The target is a whole device, a hardware partition, a partition by >> number or name, and an optional byte offset into it. >> >> Patch 1 fixes the buffer size check for a reply without Content-Length; >> patch 4 sets that check to the target size, and without the fix such a >> reply would be refused. Patch 2 lets a wget client take the data >> through a callback, patch 3 adds the block device writer, and patch 4 >> adds the options to the command. >> >> Tested: >> - ROCK Pi 4A: a 1862270976-byte image to the SD card in 3 min 16 s to >> 3 min 35 s with a 4 MiB buffer; the first 1862270976 bytes of the >> card have the sha256 of the image, and the card boots. A >> 2936012800-byte image, past the Content-Length limit below, took >> 4 min 18 s and checks and boots the same way. >> - QEMU arm64, virtio-blk: a 404 writes nothing; a file larger than the >> target is refused before any write with Content-Length, and at the >> end of the target without it. >> - sandbox: ut lib lib_test_sink_blk; every patch builds on sandbox and >> sandbox64_lwip. >> >> Notes: >> - lwIP's http_client.c reads Content-Length through an int, so from >> 2 GiB on the length is unknown and a file that does not fit fails >> only at the end of the target. >> - EFI HTTP, the other user of check_buffer_size, retries with the >> reported length when it exceeds the buffer (efi_net.c:1715), so a >> reply without Content-Length still ends in a 4 GiB allocation there; >> patch 1 does not change that. >> - checkpatch asks for IS_ENABLED() at "#if defined(CONFIG_WGET_STREAM)" >> in cmd/lwip/wget.c; that #if picks lines of the help string, as the >> WGET_CACERT ones next to it do. >> >> gzip and zstd input follow in a separate series on top of this one. >> >> Signed-off-by: Igor Velkov <[email protected]> >> --- >> Igor Velkov (4): >> net: lwip: wget: accept a reply without Content-Length in the size >> check >> net: lwip: wget: hand the payload to an optional sink >> lib: add a stream sink for block devices >> cmd: lwip: wget: write the download to a block device >> >> cmd/Kconfig | 9 ++ >> cmd/lwip/wget.c | 107 +++++++++++++++++++++- >> configs/sandbox64_lwip_defconfig | 1 + >> configs/sandbox_defconfig | 1 + >> doc/usage/cmd/wget.rst | 43 ++++++++- >> include/net-common.h | 6 ++ >> include/stream_sink.h | 75 ++++++++++++++++ >> lib/Kconfig | 8 ++ >> lib/Makefile | 1 + >> lib/stream_sink.c | 189 >> +++++++++++++++++++++++++++++++++++++++ >> net/lwip/wget.c | 89 +++++++++++++++--- >> test/lib/Makefile | 3 + >> test/lib/stream_sink.c | 137 ++++++++++++++++++++++++++++ >> 13 files changed, 655 insertions(+), 14 deletions(-) >> --- >> base-commit: 3401e006432abcbf7d73d7210f10df157f990ed3 >> change-id: 20261101-b4-wget-blk-r4-833f8a3d344e >> >> Best regards, > > Applied to u-boot-net branch for-main [1], thanks! > > [1] > https://git.u-boot-project.org/u-boot/custodians/u-boot-net/-/commits/for-main > > [1/4] net: lwip: wget: accept a reply without Content-Length in the size check > commit: 8498fbf8788eff07bc7fc59f9b15cf6c65025b10 > > [2/4] net: lwip: wget: hand the payload to an optional sink > commit: 40611fe0277a7a8f9639aab02a6f1a1e0e897d57 > > [3/4] lib: add a stream sink for block devices > commit: fdf4d9621c8b507f69fe0dd09fc53ebfe860fda5 > > [4/4] cmd: lwip: wget: write the download to a block device > commit: f0870bb385cf923bfab0baddacc952151efa4c39 > > Best regards,
FYI stating my objection to making the wget command take these arguments, it seems short sighted and not very U-Boot -like. The functionality itself I have no objection to but including this command line argument complexity now for 'wget' implies that scripts and documentation will be built around this change and then we must maintain that compatibility, which is a bad design choice in my opinion. Respectfully, -E
