Re: dlang-requests 0.1.7 released

2016-06-22 Thread ikod via Digitalmars-d-announce

On Wednesday, 22 June 2016 at 02:04:25 UTC, Zekereth wrote:

On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:

Hello,

Dlang-requests is library created under influence of 
Python-requests, with primary goal of easy to use and 
performance.


I have a couple of questions. If a url can't be found and I 
catch the ConnectError exception and handle it[2] I still get 
an additional error[1] printed. For example:


try
{
getContent(url);
}
catch(ConnectError ex)
{
debug writeln("Error getting page: ", ex.msg);
}

Will error with this:

[1]2016-06-21T20:59:15.073:streams.d:connect:750 Failed to 
connect: can't resolve www.systemcontrolpanel.com - getaddrinfo 
error: Name or service not known


[2]Error getting page: Can't connect to 
www.systemcontrolpanel.com:80: getaddrinfo error: Name or 
service not known


The additional error[1] will still print in release build. The 
only workaround I could find was to check getAddress myself and 
ensure that the page exists before calling getContent.


Thanks!


Hello,

It makes no sense to log error message and throw exception in 
next line, so I'll remove this error messages from the code.


Thanks for report(but better place it on the github project page 
so that I can link code changes with bug reports)!




Re: dlang-requests 0.1.7 released

2016-06-21 Thread Zekereth via Digitalmars-d-announce

On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:

Hello,

Dlang-requests is library created under influence of 
Python-requests, with primary goal of easy to use and 
performance.


I have a couple of questions. If a url can't be found and I catch 
the ConnectError exception and handle it[2] I still get an 
additional error[1] printed. For example:


try
{
getContent(url);
}
catch(ConnectError ex)
{
debug writeln("Error getting page: ", ex.msg);
}

Will error with this:

[1]2016-06-21T20:59:15.073:streams.d:connect:750 Failed to 
connect: can't resolve www.systemcontrolpanel.com - getaddrinfo 
error: Name or service not known


[2]Error getting page: Can't connect to 
www.systemcontrolpanel.com:80: getaddrinfo error: Name or service 
not known


The additional error[1] will still print in release build. The 
only workaround I could find was to check getAddress myself and 
ensure that the page exists before calling getContent.


Thanks!





Re: dlang-requests 0.1.7 released

2016-06-20 Thread ikod via Digitalmars-d-announce

On Friday, 17 June 2016 at 17:31:55 UTC, ikod wrote:
On Tuesday, 14 June 2016 at 14:59:37 UTC, Andrei Alexandrescu 
wrote:

On 6/11/16 7:03 PM, ikod wrote:

Hello,

Dlang-requests is library created under influence of 
Python-requests,

with primary goal of easy to use and performance.


...


Thanks! Does the project have a dub presence? How does it 
compare feature-wise and speed-wise with curl? -- Andrei


Hello,

Finally, I made some improvements and run minimal performance 
tests against command-line curl. I wrote simple code for file 
download using dlang-requests, run it and curl for the same 
urls(httpbin server on my notebook) and compare "total", 
"system", and "user" time for different cases. You can find 
numbers and code below.
So my conclusion is - performances are comparable for these 
cases, but there is some field for improvement in 
dlang-requests.


Case 1 - 50Mb of random data, no any encoding
Case 2 - 50Mb of random data, transfer chunked
Case 3 - 50Mb of random data, transfer chunked, content gzip

  measured times, sec
-
 |   user|   system  |   total
 Case|---|---|---
 | d-r | curl|  d-r| curl| d-r | curl
-|-|-|-|-|-|-
  1  | 0.17| 0.14| 0.20| 0.32| 51.7| 52.2
  2  | 0.19| 0.11| 0.15| 0.21| 51.8| 51.9
  3  | 0.21| 0.15| 0.11| 0.15| 51.5| 52.1


Finally after all unnecessary memory movements were cleaned up, 
results for dlang-requests and curl almost equals.


  measured times, sec
 -
  |   user|   system  |   total
  Case|---|---|---
  | d-r | curl|  d-r| curl| d-r | curl
 -|-|-|-|-|-|-
   1  | 0.14| 0.14| 0.15| 0.32| 53.3| 52.2
   2  | 0.13| 0.11| 0.11| 0.21| 51.1| 51.9
   3  | 0.18| 0.15| 0.08| 0.15| 51.2| 52.1




import std.stdio;
import requests;

pragma(lib, "ssl");
pragma(lib, "crypto");

void main()
{
auto sink = File("/dev/null", "wb");
auto rq = Request();
rq.useStreaming = true;
auto rs = 
rq.get("http://127.0.0.1:8080/stream-bytes/5120;);

auto stream = rs.receiveAsRange();
while(!stream.empty) {
sink.rawWrite(stream.front);
stream.popFront;
}
}





Re: dlang-requests 0.1.7 released

2016-06-17 Thread ikod via Digitalmars-d-announce
On Tuesday, 14 June 2016 at 14:59:37 UTC, Andrei Alexandrescu 
wrote:

On 6/11/16 7:03 PM, ikod wrote:

Hello,

Dlang-requests is library created under influence of 
Python-requests,

with primary goal of easy to use and performance.


...


Thanks! Does the project have a dub presence? How does it 
compare feature-wise and speed-wise with curl? -- Andrei


Hello,

Finally, I made some improvements and run minimal performance 
tests against command-line curl. I wrote simple code for file 
download using dlang-requests, run it and curl for the same 
urls(httpbin server on my notebook) and compare "total", 
"system", and "user" time for different cases. You can find 
numbers and code below.
So my conclusion is - performances are comparable for these 
cases, but there is some field for improvement in dlang-requests.


Case 1 - 50Mb of random data, no any encoding
Case 2 - 50Mb of random data, transfer chunked
Case 3 - 50Mb of random data, transfer chunked, content gzip

  measured times, sec
-
 |   user|   system  |   total
 Case|---|---|---
 | d-r | curl|  d-r| curl| d-r | curl
-|-|-|-|-|-|-
  1  | 0.17| 0.14| 0.20| 0.32| 51.7| 52.2
  2  | 0.19| 0.11| 0.15| 0.21| 51.8| 51.9
  3  | 0.21| 0.15| 0.11| 0.15| 51.5| 52.1


import std.stdio;
import requests;

pragma(lib, "ssl");
pragma(lib, "crypto");

void main()
{
auto sink = File("/dev/null", "wb");
auto rq = Request();
rq.useStreaming = true;
auto rs = 
rq.get("http://127.0.0.1:8080/stream-bytes/5120;);

auto stream = rs.receiveAsRange();
while(!stream.empty) {
sink.rawWrite(stream.front);
stream.popFront;
}
}



Re: dlang-requests 0.1.7 released

2016-06-14 Thread Seb via Digitalmars-d-announce

On Tuesday, 14 June 2016 at 22:46:00 UTC, ikod wrote:
On Tuesday, 14 June 2016 at 14:59:37 UTC, Andrei Alexandrescu 
wrote:

On 6/11/16 7:03 PM, ikod wrote:

Hello,

Dlang-requests is library created under influence of

...
Code and docs available at 
https://github.com/ikod/dlang-requests or as

dub package.

Thanks Dlang authors and community for excellent language.


Thanks! Does the project have a dub presence? How does it 
compare feature-wise and speed-wise with curl? -- Andrei


Yes, it is in the dub repository under name "requests"
As for speed comparison, I'll post numbers in a day or two, 
when I'll finish some performance improvements.


Feature matrix for libcurl(per 
https://curl.haxx.se/docs/features.html) and dlang-requests: 
https://github.com/ikod/dlang-requests/blob/master/docs/matrix.txt Please note, that dlang-requests supports only http,https and ftp.


No worries - `std.net.curl` doesn't expose all the exotic 
protocols that libcurl supports either ;-)
Same goes for the features, it's more interestingly to see what 
std.net.curl exposes to the user:


http://dlang.org/phobos/std_net_curl.html


Re: dlang-requests 0.1.7 released

2016-06-14 Thread ikod via Digitalmars-d-announce
On Tuesday, 14 June 2016 at 14:59:37 UTC, Andrei Alexandrescu 
wrote:

On 6/11/16 7:03 PM, ikod wrote:

Hello,

Dlang-requests is library created under influence of

...
Code and docs available at 
https://github.com/ikod/dlang-requests or as

dub package.

Thanks Dlang authors and community for excellent language.


Thanks! Does the project have a dub presence? How does it 
compare feature-wise and speed-wise with curl? -- Andrei


Yes, it is in the dub repository under name "requests"
As for speed comparison, I'll post numbers in a day or two, when 
I'll finish some performance improvements.


Feature matrix for libcurl(per 
https://curl.haxx.se/docs/features.html) and dlang-requests: 
https://github.com/ikod/dlang-requests/blob/master/docs/matrix.txt Please note, that dlang-requests supports only http,https and ftp.





Re: dlang-requests 0.1.7 released

2016-06-14 Thread Andrei Alexandrescu via Digitalmars-d-announce

On 6/11/16 7:03 PM, ikod wrote:

Hello,

Dlang-requests is library created under influence of Python-requests,
with primary goal of easy to use and performance.

It provide interfaces for HTTP(S) and FTP requests. You can tune request
details, but for most cases you will need single and simple API call.

Latest release added streaming content from server and some performance
improvements (avoiding some of unnecessary buffer copying).

Code and docs available at https://github.com/ikod/dlang-requests or as
dub package.

Thanks Dlang authors and community for excellent language.


Thanks! Does the project have a dub presence? How does it compare 
feature-wise and speed-wise with curl? -- Andrei




Re: dlang-requests 0.1.7 released

2016-06-13 Thread ikod via Digitalmars-d-announce

On Monday, 13 June 2016 at 08:29:28 UTC, Alexander Milushev wrote:

On Monday, 13 June 2016 at 00:09:12 UTC, ikod wrote:
On Sunday, 12 June 2016 at 22:37:34 UTC, Alexander Milushev 
wrote:

On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:

[...]


Hi, this is great project, but what about vibe.d 
compatibility?


This project doesn't depend on vibe.d, and I'm not sure if it 
can be directly used from inside vibe.d application, as it use 
blocked socket io.


Yeah, I understood, but is it possible to add support of vibe.d 
sockets too?


If you want to use requests from vibe.d application then this 
probably can be done using 
https://github.com/rejectedsoftware/vibe.d/wiki#integrating-non-vibed-io
But this require writing some wrapper. I can investigate this in 
the future.


Thanks!



Re: dlang-requests 0.1.7 released

2016-06-13 Thread Alexander Milushev via Digitalmars-d-announce

On Monday, 13 June 2016 at 00:09:12 UTC, ikod wrote:
On Sunday, 12 June 2016 at 22:37:34 UTC, Alexander Milushev 
wrote:

On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:

[...]


Hi, this is great project, but what about vibe.d compatibility?


This project doesn't depend on vibe.d, and I'm not sure if it 
can be directly used from inside vibe.d application, as it use 
blocked socket io.


Yeah, I understood, but is it possible to add support of vibe.d 
sockets too?


Re: dlang-requests 0.1.7 released

2016-06-12 Thread ikod via Digitalmars-d-announce

On Sunday, 12 June 2016 at 22:37:34 UTC, Alexander Milushev wrote:

On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:

Hello,

Dlang-requests is library created under influence of 
Python-requests, with primary goal of easy to use and 
performance.


It provide interfaces for HTTP(S) and FTP requests. You can 
tune request details, but for most cases you will need single 
and simple API call.


Latest release added streaming content from server and some 
performance improvements (avoiding some of unnecessary buffer 
copying).


Code and docs available at 
https://github.com/ikod/dlang-requests or as dub package.


Thanks Dlang authors and community for excellent language.


Hi, this is great project, but what about vibe.d compatibility?


This project doesn't depend on vibe.d, and I'm not sure if it can 
be directly used from inside vibe.d application, as it use 
blocked socket io.


Re: dlang-requests 0.1.7 released

2016-06-12 Thread Alexander Milushev via Digitalmars-d-announce

On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:

Hello,

Dlang-requests is library created under influence of 
Python-requests, with primary goal of easy to use and 
performance.


It provide interfaces for HTTP(S) and FTP requests. You can 
tune request details, but for most cases you will need single 
and simple API call.


Latest release added streaming content from server and some 
performance improvements (avoiding some of unnecessary buffer 
copying).


Code and docs available at 
https://github.com/ikod/dlang-requests or as dub package.


Thanks Dlang authors and community for excellent language.


Hi, this is great project, but what about vibe.d compatibility?


Re: dlang-requests 0.1.7 released

2016-06-12 Thread Suliman via Digitalmars-d-announce

Voting for making requests replacement of curl!