On Sunday, 5 April 2020 at 11:53:29 UTC, Petar Kirov [ZombineDev]
wrote:
On Sunday, 5 April 2020 at 08:59:50 UTC, ikod wrote:
Hello!
Just a note that dlang-requests ver 1.1.0 released with new
'ByLine' interfaces added for get/post/put requests.
range algorithms can be applied to server responses, so that
simple chain
getContentByLine("https://httpbin.org/anything";)
.map!"cast(string)a"
.filter!(a => a.canFind("data"))
should work.
These calls work lazily so you can apply them to large
documents.
dlang-requests - HTTP client library, inspired by
python-requests with goals:
small memory footprint
performance
simple, high level API
native D implementation
https://github.com/ikod/dlang-requests
https://code.dlang.org/packages/requests
Always waiting for your bugreports and proposals on project
page.
Best regards!
Nice work!
One quick suggestion: avoid direct casting from `ubyte[]` to
`string`:
/+dub.sdl:
dependency "requests" version="~>1.1"
+/
void main()
{
import requests : getContentByLine;
import std : assumeUTF, canFind, each, filter, map, write;
getContentByLine("https://httpbin.org/anything";)
.map!assumeUTF // instead of map!"cast(string)a"
.filter!(a => a.canFind("data"))
.each!write;
}
1. From a code-style point of view, assumeUTF is better as it
shows the intention to the reader - assume that the content is
valid UTF8 encoded text, without performing validation. And if
there are UTF8 errors, it is easy to go back and add validation
there.
2. Avoid casting mutable data to immutable. The data path in
your library is rather complex (getContentByLine -> _LineReader
-> LineSplitter -> Buffer -> ...) and so it was hard to
understand from a quick glance whether or not the buffer array
is reused (but I would guess that it is). If the buffer array
is reused, it means that the result of calling
_LineReader.front() may be modified at a later point in time,
which I think is obvious that it can lead to some rather nasty
bugs in users' code.
Internally my code do not modify these data, but I understand
your concern.
I suggest you look into Steven's iopipe[1] library, as I
believe it can help you clean up and refactor this part of the
codebase (and can probably yield some performance improvements
along the way).
[1]: https://github.com/schveiguy/iopipe
Thanks! You are totally right regarding casting mutable to
immutable. Base code was written when I did not understand this
problem well enough. I know how to fix this but this probably
require large enough rewrite. I hope I'll do this in some near
future.
PS. I also have package which convert nogc unique mutable byte
array to immutable byte array and attach this immutable data "as
is" to buffer (which is essentally array of immutable(ubyte)[]).
Then every operation on buffer (trim, split, search, etc)
performed on immutable with zero copy and with warranties on data
immutability.
