Hello,
  I unfortunately don't know `libcurl` sources well, and I program primarily 
for devices with huge memory available, so in my context it is not a big issue 
to increase the limit; if we have 128 GB of memory, why not to make 1 megabyte 
buffer? Of course, there is possibility that we'll have something unusually 
huge, but in 99,99% of situation just increasing the buffer is enough, and if 
data size will be even more than huge buffer, it is almost for sure error 
situation, so `libcurl` may just fail with some error code. As a summary:

  1. We add `CULOPT_BUFFER_SIZE` where we can overwrite the buffer size, or we 
may even supply own buffer pointer to `libcurl` as a pointer to structure that 
contains size and data fields.
  2. If frame is even bigger, `libcurl` generates buffer overflow error, or 
calls any sort of callback that allows custom program to decide what to do: 
increase the buffer or cancel the operation, something like this:

  bool OnBufferOverflow(size_t new_size);

  and if we return `true`, `libcurl` has to increase the buffer, and it has to 
cancel the operation if we return `false`.

  3. We make this mode optional and disabled by default to not affect existing 
applications or applications that expect huge data amount and don't want to 
load it in memory.  

  `curl_ws_meta` is also good solution, even it is not fully automated. May be 
just update documentation is good idea. For me the entry point to websocket 
support in curl was this page:

  https://curl.se/docs/websocket.html
 
  Google gives it when you search "libcurl websocket", so it is logical entry 
point for new protocol users. I see here you mention `curl_ws_meta`, but I 
simply missed it. My logic was like that: "Well, meta is for low level 
programmers, and I just need the stream of messages, so I can skip this 
function.". May be link to any working example of write function with 
`curl_ws_meta` call, or any highlight that this function is very important and 
that without it you will most probably not receive the frames stream like you 
expect them will help other `libcurl` users to investigate more effectively.

  Actually idea of simple "write function" that just gives you infinite stream 
of data is very good for applications that just receive streams of data, 
including trading, social media monitoring, etc..., and updating API to 
increase `libcurl` buffer size, or to supply custom buffer will simplify life 
for many users that don't need anything complex and just need stream of data. 
There are possibility that we'll get unusually huge payload, but it is for sure 
not mainstream situation; unlike HTTP that we sometimes use to download huge 
files, websocket is realtime protocol for fast exchange of small messages, so 
simplification of API is very good idea.
--
Best regards,
  Vitalii B. Avramenko.

-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Reply via email to