On Sunday, 5 April 2020 at 08:59:50 UTC, ikod wrote:
Hello!
Just a note that dlang-requests ver 1.1.0 released with new
'ByLine' interfaces added for get/post/put requests.
range algorithms can be applied to server responses, so that
simple chain
getContentByLine("https://httpbin.org/anything";)
.map!"cast(string)a"
.filter!(a => a.canFind("data"))
should work.
These calls work lazily so you can apply them to large
documents.
dlang-requests - HTTP client library, inspired by
python-requests with goals:
small memory footprint
performance
simple, high level API
native D implementation
https://github.com/ikod/dlang-requests
https://code.dlang.org/packages/requests
Always waiting for your bugreports and proposals on project
page.
Best regards!
Nice work!
One quick suggestion: avoid direct casting from `ubyte[]` to
`string`:
/+dub.sdl:
dependency "requests" version="~>1.1"
+/
void main()
{
import requests : getContentByLine;
import std : assumeUTF, canFind, each, filter, map, write;
getContentByLine("https://httpbin.org/anything";)
.map!assumeUTF // instead of map!"cast(string)a"
.filter!(a => a.canFind("data"))
.each!write;
}
1. From a code-style point of view, assumeUTF is better as it
shows the intention to the reader - assume that the content is
valid UTF8 encoded text, without performing validation. And if
there are UTF8 errors, it is easy to go back and add validation
there.
2. Avoid casting mutable data to immutable. The data path in your
library is rather complex (getContentByLine -> _LineReader ->
LineSplitter -> Buffer -> ...) and so it was hard to understand
from a quick glance whether or not the buffer array is reused
(but I would guess that it is). If the buffer array is reused, it
means that the result of calling _LineReader.front() may be
modified at a later point in time, which I think is obvious that
it can lead to some rather nasty bugs in users' code.
I suggest you look into Steven's iopipe[1] library, as I believe
it can help you clean up and refactor this part of the codebase
(and can probably yield some performance improvements along the
way).
[1]: https://github.com/schveiguy/iopipe
