Re: std.net.curl and HTTP.responseHeaders

2021-02-04 Thread Vindex via Digitalmars-d-learn

Thank you!
For other sites, the first solution somehow worked (I did it 
following the example from the documentation).


Re: std.net.curl and HTTP.responseHeaders

2021-02-04 Thread Anonymouse via Digitalmars-d-learn

On Wednesday, 3 February 2021 at 19:25:18 UTC, Vindex wrote:

Output:
```
std.net.curl.HTTPStatusException@/usr/include/dmd/phobos/std/net/curl.d(1097): 
HTTP request returned status code 405 ()
```

Perhaps some special HTTP configuration is needed?


Is this closer to what you want?

import std.stdio, std.net.curl;

void main() {
enum url = "https://en.wikipedia.org/wiki/Passenger_pigeon;;
auto http = HTTP(url);
http.perform();
writeln(http.responseHeaders);
}


Re: std.net.curl : Performance

2020-11-09 Thread Vino via Digitalmars-d-learn

On Monday, 9 November 2020 at 20:57:33 UTC, Daniel Kozak wrote:
On Mon, Nov 9, 2020 at 9:50 PM rinfz via Digitalmars-d-learn < 
digitalmars-d-learn@puremagic.com> wrote:



On Monday, 9 November 2020 at 20:40:59 UTC, rinfz wrote:
> On Monday, 9 November 2020 at 19:55:07 UTC, Vino wrote:
>> ...
>
> The only curl option you need to set within the loop is the 
> CurlOption.url. So your foreach block should look more like:

>
> foreach (...) {
> string url = chain(apihost, only(':'), 
> to!string(apiport),

> apiuri).to!string;
> https.handle.set(CurlOption.url, url);
> https.perform();
> scope(failure) exit(-4);
> scope(exit) https.shutdown;
> apidata.insert(tuple(seq, cast(string) content));
> content = [];
> }
>
> Every other line can be placed before the foreach.

In fact, you don't need url in there either since it's not 
dependent on loop variables, and you don't need the scope 
guards if this is running in main (you can move them out of 
the loop too).


foreach (...) {
 https.handle.set(CurlOption.url, url);
 https.perform();
 apidata.insert(tuple(seq, cast(string) content));
 content = [];
}



In fact he does not need foreach. Because he use it on empty 
result


Hi All,

  The reason that the above code is within foreach loop is 
because there are several api's and each of the api's have 
separate configuration. These configuration are maintained in a 
table in MySQL database, the result contains the configuration 
details for each of these api's, below is the complete code.


Sequence
The code collects the data from several api's whose output is 
json.
The output of the api's are parsed, filtered, validated and 
stored in a array using another function.(using asdf json parser)
The data stores in array is then inserted into the table in MySQL 
database.(using hunt-database).


Code:
auto getData ()
{
 Array!(Tuple!(int,string)) apidata;
 Row[] result;
 result = getApiconf();  \\ fetch's the api configuration 
from the database table.

 foreach(i, k; parallel(result,1))
{

  string apihost = result[i][0].get!(string);
  int apiport = result[i][1].get!(int);
  string apiuser = result[i][2].get!(string);
  string apipass = result[i][3].get!(string);
  int seq = result[i][4].get!int;
  string apiuri = result[i][5].get!(string);
  int connecttimeout = result[i][6].get!(int);
  int tcp_nodelay = result[i][7].get!(int);
  int http_version = result[i][8].get!(int);
  int sslversion = result[i][9].get!(int);
  int use_ssl = result[i][10].get!(int);
  int ssl_verifypeer = result[i][11].get!(int);

  string url = chain(apihost, only(':'), to!string(apiport), 
apiuri).to!string;
  string usrpass = chain(apiuser, only(':'), 
apipass).to!string;


  auto https = HTTP();
  https.handle.set(CurlOption.buffersize, 512000);
  https.handle.set(CurlOption.userpwd, usrpass);
  https.handle.set(CurlOption.connecttimeout, connecttimeout);
  https.handle.set(CurlOption.tcp_nodelay, tcp_nodelay);
  https.handle.set(CurlOption.http_version, http_version);
  https.handle.set(CurlOption.sslversion,  sslversion);
  https.handle.set(CurlOption.use_ssl,  use_ssl);
  https.handle.set(CurlOption.ssl_verifypeer, ssl_verifypeer);
  https.handle.set(CurlOption.url, url);
  https.method(HTTP.Method.get);
  https.StatusLine st;
  https.onReceiveStatusLine = (https.StatusLine st) { if 
(st.code != 200)

 {
   throw new Exception(st.reason);
 } };
  ubyte[] content;
  https.onReceive = (ubyte[] data) { content ~= data; return 
data.length; };

  https.perform();
  scope(failure) { https.shutdown; exit(-4); } scope(exit) 
https.shutdown;

  apidata.insert(tuple(seq, cast(string) content));
 }
 return apidata[].sort;
}

 From,
Vino.B



Re: std.net.curl : Performance

2020-11-09 Thread Daniel Kozak via Digitalmars-d-learn
On Mon, Nov 9, 2020 at 9:50 PM rinfz via Digitalmars-d-learn <
digitalmars-d-learn@puremagic.com> wrote:

> On Monday, 9 November 2020 at 20:40:59 UTC, rinfz wrote:
> > On Monday, 9 November 2020 at 19:55:07 UTC, Vino wrote:
> >> ...
> >
> > The only curl option you need to set within the loop is the
> > CurlOption.url. So your foreach block should look more like:
> >
> > foreach (...) {
> > string url = chain(apihost, only(':'), to!string(apiport),
> > apiuri).to!string;
> > https.handle.set(CurlOption.url, url);
> > https.perform();
> > scope(failure) exit(-4);
> > scope(exit) https.shutdown;
> > apidata.insert(tuple(seq, cast(string) content));
> > content = [];
> > }
> >
> > Every other line can be placed before the foreach.
>
> In fact, you don't need url in there either since it's not
> dependent on loop variables, and you don't need the scope guards
> if this is running in main (you can move them out of the loop
> too).
>
> foreach (...) {
>  https.handle.set(CurlOption.url, url);
>  https.perform();
>  apidata.insert(tuple(seq, cast(string) content));
>  content = [];
> }
>

In fact he does not need foreach. Because he use it on empty result


Re: std.net.curl : Performance

2020-11-09 Thread rinfz via Digitalmars-d-learn

On Monday, 9 November 2020 at 20:40:59 UTC, rinfz wrote:

On Monday, 9 November 2020 at 19:55:07 UTC, Vino wrote:

...


The only curl option you need to set within the loop is the 
CurlOption.url. So your foreach block should look more like:


foreach (...) {
string url = chain(apihost, only(':'), to!string(apiport), 
apiuri).to!string;

https.handle.set(CurlOption.url, url);
https.perform();
scope(failure) exit(-4);
scope(exit) https.shutdown;
apidata.insert(tuple(seq, cast(string) content));
content = [];
}

Every other line can be placed before the foreach.


In fact, you don't need url in there either since it's not 
dependent on loop variables, and you don't need the scope guards 
if this is running in main (you can move them out of the loop 
too).


foreach (...) {
https.handle.set(CurlOption.url, url);
https.perform();
apidata.insert(tuple(seq, cast(string) content));
content = [];
}


Re: std.net.curl : Performance

2020-11-09 Thread rinfz via Digitalmars-d-learn

On Monday, 9 November 2020 at 19:55:07 UTC, Vino wrote:

...


The only curl option you need to set within the loop is the 
CurlOption.url. So your foreach block should look more like:


foreach (...) {
string url = chain(apihost, only(':'), to!string(apiport), 
apiuri).to!string;

https.handle.set(CurlOption.url, url);
https.perform();
scope(failure) exit(-4);
scope(exit) https.shutdown;
apidata.insert(tuple(seq, cast(string) content));
content = [];
}

Every other line can be placed before the foreach.



Re: std.net.curl : Performance

2020-11-09 Thread Daniel Kozak via Digitalmars-d-learn
Just delete it

On Mon, Nov 9, 2020 at 9:00 PM Vino via Digitalmars-d-learn <
digitalmars-d-learn@puremagic.com> wrote:

> Hi All,
>
>Request your help to on how to improve the performance of the
> below code.
>
> import std.conv: to;
> import std.net.curl : get, HTTP, CurlOption;
> import std.parallelism: parallel;
> import std.range: chain, only;
> import std.typecons: Tuple, tuple;
>
> void main ()
> {
>   Array!(Tuple!(int,string)) apidata;
>   Row[] result;
>   string apihost = "abc.com"; int apiport = 1830; string apiuri =
> /getdata;
>   string apiuser = "user"; string apipass = "pass";
>   foreach(i, k; parallel(result,1))
>  {
>string url = chain(apihost, only(':'), to!string(apiport),
> apiuri).to!string;
>string usrpass = chain(apiuser, only(':'),
> apipass).to!string;
>auto https = HTTP();
>https.handle.set(CurlOption.buffersize, 512000);
>https.handle.set(CurlOption.userpwd, usrpass);
>https.handle.set(CurlOption.connecttimeout, 600);
>https.handle.set(CurlOption.tcp_nodelay, 1);
>https.handle.set(CurlOption.http_version, 2);
>https.handle.set(CurlOption.sslversion,  1;
>https.handle.set(CurlOption.use_ssl,  3);
>https.handle.set(CurlOption.ssl_verifypeer, 0);
>https.handle.set(CurlOption.url, url);
>https.method(HTTP.Method.get);
>https.StatusLine st;
>https.onReceiveStatusLine = (https.StatusLine st) { if
> (st.code != 200) { throw new Exception(st.reason); } };
>ubyte[] content;
>https.onReceive = (ubyte[] data) { content ~= data; return
> data.length; };
>https.perform();
>scope(failure) { https.shutdown; exit(-4); } scope(exit)
> https.shutdown;
>apidata.insert(tuple(seq, cast(string) content));
>   }
>   return apidata[].sort;
> }
>
> From,
> Vino.B
>


Re: std.net.curl get json_encode

2020-10-11 Thread Andre Pany via Digitalmars-d-learn

On Sunday, 11 October 2020 at 08:48:16 UTC, Vino wrote:

On Friday, 9 October 2020 at 17:50:16 UTC, Andre Pany wrote:

[...]


Hi Andre,

  Thank you very much, now we are able to get the data as 
expected using jv["Name"], now when we try to print all the 
returned data with Key and Values as below it is thorwing an 
error


Error:

Error: template `object.byKeyValue` cannot deduce function from 
argument types `!()(JSONValue)`


Code:

string s = cast(string) content;
JSONValue jv = parseJSONValue(s);
writeln(jv["Name"];   \\ this works.
foreach (key, value; jv.byKeyValue) writefln("%s: %s", key, 
value);  \\ this does not work.


From,
Vino


As far as I remember:
foreach (key, value; jv.object.byKeyValue)

byKeyValue is a function of an associative array. You get the 
associative array (Json object) of an JSONValue by attribute 
.object.


Kind regards
Andre


Re: std.net.curl get json_encode

2020-10-11 Thread Vino via Digitalmars-d-learn

On Friday, 9 October 2020 at 17:50:16 UTC, Andre Pany wrote:

On Friday, 9 October 2020 at 05:56:05 UTC, Vino wrote:

On Friday, 9 October 2020 at 05:30:34 UTC, ikod wrote:

On Friday, 9 October 2020 at 01:45:37 UTC, Vino wrote:

On Friday, 2 October 2020 at 23:20:48 UTC, Imperatorn wrote:

On Friday, 2 October 2020 at 21:12:09 UTC, Vino wrote:

Hi All,

...

auto content = https.perform();
https.shutdown;
JSONValue jv = parseJSONValue(content);


Maybe
JSONValue jv = toJSONValue(content);



writeln(jv["Name"]);

}

From,
Vino


Hi Ikod,

  No luck.

Error
test.d(19): Error: template 
`stdx.data.json.parser.toJSONValue` cannot deduce function 
from argument types `!()(int)`, candidates are:

C:\D\dmd2\windows\bin\..\..\src\phobos\stdx\data\json\parser.d(58):
`toJSONValue(LexOptions options = LexOptions.init, Input)(Input input, string filename = 
"", int maxDepth = defaultMaxDepth)`
  with `options = cast(LexOptions)0,
   Input = int`
  whose parameters have the following constraints:
  ``
`  > isInputRange!Input
  - isSomeChar!(ElementType!Input)
or:
  - isIntegral!(ElementType!Input)
`  ``
C:\D\dmd2\windows\bin\..\..\src\phobos\stdx\data\json\parser.d(65):
`toJSONValue(Input)(Input tokens, int maxDepth = defaultMaxDepth)`
  with `Input = int`
  whose parameters have the following constraints:
  ``
`  > isJSONTokenInputRange!Input
`  ``
datacoll.d(19):All possible candidates are marked as 
`deprecated` or `@disable`

  Tip: not satisfied constraints are marked with `>`

From,
Vino


(Writing from my mobile, a lot of room for improvements):

import std.net.curl, std.stdio;

// GET with custom data receivers
auto http = HTTP("dlang.org");
http.onReceiveHeader =
(in char[] key, in char[] value) { writeln(key, ": ", 
value); };


ubyte[] content;
http.onReceive = (ubyte[] data) {
content ~= data;
return data.length; };
http.perform();

string s = cast(string) content;

Kind regards
Andre


Hi Andre,

  Thank you very much, now we are able to get the data as 
expected using jv["Name"], now when we try to print all the 
returned data with Key and Values as below it is thorwing an error


Error:

Error: template `object.byKeyValue` cannot deduce function from 
argument types `!()(JSONValue)`


Code:

string s = cast(string) content;
JSONValue jv = parseJSONValue(s);
writeln(jv["Name"];   \\ this works.
foreach (key, value; jv.byKeyValue) writefln("%s: %s", key, 
value);  \\ this does not work.


From,
Vino


Re: std.net.curl get json_encode

2020-10-09 Thread Andre Pany via Digitalmars-d-learn

On Friday, 9 October 2020 at 05:56:05 UTC, Vino wrote:

On Friday, 9 October 2020 at 05:30:34 UTC, ikod wrote:

On Friday, 9 October 2020 at 01:45:37 UTC, Vino wrote:

On Friday, 2 October 2020 at 23:20:48 UTC, Imperatorn wrote:

On Friday, 2 October 2020 at 21:12:09 UTC, Vino wrote:

Hi All,

...

auto content = https.perform();
https.shutdown;
JSONValue jv = parseJSONValue(content);


Maybe
JSONValue jv = toJSONValue(content);



writeln(jv["Name"]);

}

From,
Vino


Hi Ikod,

  No luck.

Error
test.d(19): Error: template `stdx.data.json.parser.toJSONValue` 
cannot deduce function from argument types `!()(int)`, 
candidates are:

C:\D\dmd2\windows\bin\..\..\src\phobos\stdx\data\json\parser.d(58):
`toJSONValue(LexOptions options = LexOptions.init, Input)(Input input, string filename = 
"", int maxDepth = defaultMaxDepth)`
  with `options = cast(LexOptions)0,
   Input = int`
  whose parameters have the following constraints:
  ``
`  > isInputRange!Input
  - isSomeChar!(ElementType!Input)
or:
  - isIntegral!(ElementType!Input)
`  ``
C:\D\dmd2\windows\bin\..\..\src\phobos\stdx\data\json\parser.d(65):
`toJSONValue(Input)(Input tokens, int maxDepth = defaultMaxDepth)`
  with `Input = int`
  whose parameters have the following constraints:
  ``
`  > isJSONTokenInputRange!Input
`  ``
datacoll.d(19):All possible candidates are marked as 
`deprecated` or `@disable`

  Tip: not satisfied constraints are marked with `>`

From,
Vino


(Writing from my mobile, a lot of room for improvements):

import std.net.curl, std.stdio;

// GET with custom data receivers
auto http = HTTP("dlang.org");
http.onReceiveHeader =
(in char[] key, in char[] value) { writeln(key, ": ", value); 
};


ubyte[] content;
http.onReceive = (ubyte[] data) {
content ~= data;
return data.length; };
http.perform();

string s = cast(string) content;

Kind regards
Andre



Re: std.net.curl get json_encode

2020-10-09 Thread Vino via Digitalmars-d-learn

On Friday, 9 October 2020 at 05:30:34 UTC, ikod wrote:

On Friday, 9 October 2020 at 01:45:37 UTC, Vino wrote:

On Friday, 2 October 2020 at 23:20:48 UTC, Imperatorn wrote:

On Friday, 2 October 2020 at 21:12:09 UTC, Vino wrote:

Hi All,

...

auto content = https.perform();
https.shutdown;
JSONValue jv = parseJSONValue(content);


Maybe
JSONValue jv = toJSONValue(content);



writeln(jv["Name"]);

}

From,
Vino


Hi Ikod,

  No luck.

Error
test.d(19): Error: template `stdx.data.json.parser.toJSONValue` 
cannot deduce function from argument types `!()(int)`, candidates 
are:

C:\D\dmd2\windows\bin\..\..\src\phobos\stdx\data\json\parser.d(58):
`toJSONValue(LexOptions options = LexOptions.init, Input)(Input input, string filename = 
"", int maxDepth = defaultMaxDepth)`
  with `options = cast(LexOptions)0,
   Input = int`
  whose parameters have the following constraints:
  ``
`  > isInputRange!Input
  - isSomeChar!(ElementType!Input)
or:
  - isIntegral!(ElementType!Input)
`  ``
C:\D\dmd2\windows\bin\..\..\src\phobos\stdx\data\json\parser.d(65):
`toJSONValue(Input)(Input tokens, int maxDepth = defaultMaxDepth)`
  with `Input = int`
  whose parameters have the following constraints:
  ``
`  > isJSONTokenInputRange!Input
`  ``
datacoll.d(19):All possible candidates are marked as 
`deprecated` or `@disable`

  Tip: not satisfied constraints are marked with `>`

From,
Vino


Re: std.net.curl get json_encode

2020-10-08 Thread ikod via Digitalmars-d-learn

On Friday, 9 October 2020 at 01:45:37 UTC, Vino wrote:

On Friday, 2 October 2020 at 23:20:48 UTC, Imperatorn wrote:

On Friday, 2 October 2020 at 21:12:09 UTC, Vino wrote:

Hi All,

...

auto content = https.perform();
https.shutdown;
JSONValue jv = parseJSONValue(content);


Maybe
JSONValue jv = toJSONValue(content);



writeln(jv["Name"]);

}

From,
Vino





Re: std.net.curl get json_encode

2020-10-08 Thread Vino via Digitalmars-d-learn

On Friday, 2 October 2020 at 23:20:48 UTC, Imperatorn wrote:

On Friday, 2 October 2020 at 21:12:09 UTC, Vino wrote:

Hi All,

   Request your help, the below code is working but we need 
the output as a json array, in PHP we have 
json_encode(content), so how to do the same in D, the output 
is as below, as we need to store this output into database 
table which contains columns' (Id, Hostname, 
pool,email_id,username)


[...]


I also suggest you take a look at other "json packages" as well:
https://code.dlang.org/search?q=json

For example std_data_json or asdf


Hi,

 I tired sdt_data_json, still no luck, the output of the url is 
as below


Output : {"Name":"testuser","Site":"1-east"}

Error :
test.d(20): Error: template 
`stdx.data.json.parser.parseJSONValue` cannot deduce function 
from argument types `!()(int)`, candidates are:

C:\D\dmd2\windows\bin\..\..\src\phobos\stdx\data\json\parser.d(133):
`parseJSONValue(LexOptions options = LexOptions.init, Input)(ref Input input, string 
filename = "", int maxDepth = defaultMaxDepth)`
  with `options = cast(LexOptions)0,
   Input = int`
  whose parameters have the following constraints:
  ``
`  > isInputRange!Input
  - isSomeChar!(ElementType!Input)
or:
  - isIntegral!(ElementType!Input)
`  ``
C:\D\dmd2\windows\bin\..\..\src\phobos\stdx\data\json\parser.d(178):
`parseJSONValue(Input)(ref Input tokens, int maxDepth = defaultMaxDepth)`
  with `Input = int`
  whose parameters have the following constraints:
  ``
`  > isJSONTokenInputRange!Input
`  ``
datacoll.d(20):All possible candidates are marked as 
`deprecated` or `@disable`

  Tip: not satisfied constraints are marked with `>`

Code:
import std.net.curl, std.conv, std.stdio, stdx.data.json;


void main() {
auto https = HTTP();
https.handle.set(CurlOption.userpwd, "user:pass");
https.handle.set(CurlOption.connecttimeout, 600);
https.handle.set(CurlOption.tcp_nodelay, 1);
https.handle.set(CurlOption.buffersize, 1073741824);
https.handle.set(CurlOption.http_version, 2);
https.handle.set(CurlOption.sslversion,  1);
https.handle.set(CurlOption.use_ssl,  3);
https.handle.set(CurlOption.ssl_verifypeer, 0);
https.handle.set(CurlOption.url, "https://test.com/getUser;);
https.method(HTTP.Method.get);
https.StatusLine status;
auto content = https.perform();
https.shutdown;
JSONValue jv = parseJSONValue(content);
writeln(jv["Name"]);

}

From,
Vino


Re: std.net.curl get json_encode

2020-10-02 Thread Imperatorn via Digitalmars-d-learn

On Friday, 2 October 2020 at 21:12:09 UTC, Vino wrote:

Hi All,

   Request your help, the below code is working but we need the 
output as a json array, in PHP we have json_encode(content), so 
how to do the same in D, the output is as below, as we need to 
store this output into database table which contains columns' 
(Id, Hostname, pool,email_id,username)


[...]


I also suggest you take a look at other "json packages" as well:
https://code.dlang.org/search?q=json

For example std_data_json or asdf


Re: std.net.curl get json_encode

2020-10-02 Thread Imperatorn via Digitalmars-d-learn

On Friday, 2 October 2020 at 21:12:09 UTC, Vino wrote:

Hi All,

   Request your help, the below code is working but we need the 
output as a json array, in PHP we have json_encode(content), so 
how to do the same in D, the output is as below, as we need to 
store this output into database table which contains columns' 
(Id, Hostname, pool,email_id,username)


[...]


JSONValue jv = parseJSON(content);


Re: std.net.curl and libcurl.so

2016-09-24 Thread Joseph Rushton Wakeling via Digitalmars-d-learn
On Saturday, 24 September 2016 at 19:42:11 UTC, Joseph Rushton 
Wakeling wrote:
On Saturday, 24 September 2016 at 19:27:31 UTC, Joseph Rushton 
Wakeling wrote:
Further to earlier remarks: I now think this may be a general 
problem of LDC 1.0.0 and not a problem of the snap package.


I tried building my simple curl-using program using an LDC 
1.0.0 build and installed from source in the standard cmake && 
make && make install fashion.  The same segfault occurs.


More on this: ldc 0.17.1 (the version packaged with Ubuntu 
16.04) is based on dmd 2.068.2, which still linked against 
libcurl.  It's only from v2.069.0+ that libcurl is loaded 
dynamically:

https://dlang.org/changelog/2.069.0.html#curl-dynamic-loading

ldc 1.0.0 is based on 2.070.2.


Downloaded a pre-built copy of ldc 1.0.0 from here (I used the 
Linux x86_64 tarball):

https://github.com/ldc-developers/ldc/releases/tag/v1.0.0

... and this runs my little curl-based program without any 
trouble.  Maybe something about how the build was carried out?


Re: std.net.curl and libcurl.so

2016-09-24 Thread Joseph Rushton Wakeling via Digitalmars-d-learn
On Saturday, 24 September 2016 at 19:27:31 UTC, Joseph Rushton 
Wakeling wrote:
Further to earlier remarks: I now think this may be a general 
problem of LDC 1.0.0 and not a problem of the snap package.


I tried building my simple curl-using program using an LDC 
1.0.0 build and installed from source in the standard cmake && 
make && make install fashion.  The same segfault occurs.


More on this: ldc 0.17.1 (the version packaged with Ubuntu 16.04) 
is based on dmd 2.068.2, which still linked against libcurl.  
It's only from v2.069.0+ that libcurl is loaded dynamically:

https://dlang.org/changelog/2.069.0.html#curl-dynamic-loading

ldc 1.0.0 is based on 2.070.2.


Re: std.net.curl and libcurl.so

2016-09-24 Thread Joseph Rushton Wakeling via Digitalmars-d-learn
On Saturday, 24 September 2016 at 19:11:52 UTC, Joseph Rushton 
Wakeling wrote:

On Friday, 23 September 2016 at 00:55:43 UTC, Stefan Koch wrote:

This suggests that libcurl is loaded.
could you compile with -g ?
and then post the output ?


Further to earlier remarks: I now think this may be a general 
problem of LDC 1.0.0 and not a problem of the snap package.


I tried building my simple curl-using program using an LDC 1.0.0 
build and installed from source in the standard cmake && make && 
make install fashion.  The same segfault occurs.


Re: std.net.curl and libcurl.so

2016-09-24 Thread Joseph Rushton Wakeling via Digitalmars-d-learn

On Friday, 23 September 2016 at 00:55:43 UTC, Stefan Koch wrote:

This suggests that libcurl is loaded.
could you compile with -g ?
and then post the output ?


Thanks Stefan!  It was compiled with -g, but I was missing the 
libcurl3-dbg package.  Here's the results:


#0  0x0046b048 in gc.gc.Gcx.smallAlloc(ubyte, ref ulong, 
uint) ()
#1  0x0046918a in gc.gc.GC.malloc(ulong, uint, ulong*, 
const(TypeInfo)) ()

#2  0x004682bc in gc_qalloc ()
#3  0x0046293b in core.memory.GC.qalloc(ulong, uint, 
const(TypeInfo)) ()
#4  0x004534f2 in 
std.array.Appender!(char[]).Appender.ensureAddable(ulong) ()
#5  0x004530c5 in 
std.uni.toCase!(std.uni.toLowerIndex(dchar), 1043, 
std.uni.toLowerTab(ulong), 
char[]).toCase(char[]).__foreachbody2(ref ulong, ref dchar) ()

#6  0x0046dc96 in _aApplycd2 ()
#7  0x0044d997 in 
std.net.curl.HTTP.Impl.onReceiveHeader(void(const(char[]), 
const(char[])) delegate).__lambda2(const(char[])) ()
#8  0x00451d78 in 
std.net.curl.Curl._receiveHeaderCallback(const(char*), ulong, 
ulong, void*) ()
#9  0x76f75a1d in Curl_client_chop_write 
(conn=conn@entry=0x741fd0, type=type@entry=2,
ptr=0x72d3a0 "Server: Apache/2.4.17 (FreeBSD) OpenSSL/1.0.2d 
PHP/5.6.16\r\n", len=59) at sendf.c:454
#10 0x76f75b9d in Curl_client_write 
(conn=conn@entry=0x741fd0, type=type@entry=2,

ptr=, len=) at sendf.c:511
#11 0x76f73f90 in Curl_http_readwrite_headers 
(data=data@entry=0x72fb60,

conn=conn@entry=0x741fd0, nread=nread@entry=0x7fffd370,
stop_reading=stop_reading@entry=0x7fffd36f) at http.c:3739
#12 0x76f8be30 in readwrite_data (done=0x7fffd40b, 
didwhat=, k=0x72fbd8,

conn=0x741fd0, data=0x72fb60) at transfer.c:492
#13 Curl_readwrite (conn=0x741fd0, data=data@entry=0x72fb60, 
done=done@entry=0x7fffd40b)

at transfer.c:1074
#14 0x76f95f42 in multi_runsingle 
(multi=multi@entry=0x7388b0, now=...,

data=data@entry=0x72fb60) at multi.c:1544
#15 0x76f96ddd in curl_multi_perform 
(multi_handle=multi_handle@entry=0x7388b0,
running_handles=running_handles@entry=0x7fffd5a4) at 
multi.c:1821
#16 0x76f8d84b in easy_transfer (multi=0x7388b0) at 
easy.c:724

#17 easy_perform (events=false, data=0x72fb60) at easy.c:812
#18 curl_easy_perform (easy=0x72fb60) at easy.c:831
#19 0x0044f971 in 
std.net.curl.HTTP.perform(std.typecons.Flag!("throwOnError").Flag) ()
#20 0x0040db01 in 
std.net.curl._basicHTTP!(char)._basicHTTP(const(char)[], 
const(void)[], std.net.curl.HTTP) ()
#21 0x004031ff in std.net.curl.get!(std.net.curl.HTTP, 
char).get(const(char)[], std.net.curl.HTTP) ()
#22 0x00403020 in 
std.net.curl.get!(std.net.curl.AutoProtocol, 
char).get(const(char)[], std.net.curl.AutoProtocol) ()

#23 0x00402f98 in D main ()



Also the call seems to fail during allocation.


Yup.  And that allocation seems to occur during use of an 
Appender while converting unicode text to lowercase ... ?


Re: std.net.curl and libcurl.so

2016-09-22 Thread Stefan Koch via Digitalmars-d-learn
On Thursday, 22 September 2016 at 23:19:27 UTC, Joseph Rushton 
Wakeling wrote:


The segfault would suggest to me that either the loading of the 
library fails or that there's some resource phobos expects to 
find which it can't access.  Can anyone advise what could be 
going on here?

-- Joe



/usr/lib/x86_64-linux-gnu/libcurl.so
#13 0x76f97986 in curl_multi_perform () from 
/usr/lib/x86_64-linux-gnu/libcurl.so
#14 0x76f8e65b in curl_easy_perform () from 
/usr/lib/x86_64-linux-gnu/libcurl.so


This suggests that libcurl is loaded.
could you compile with -g ?
and then post the output ?

Also the call seems to fail during allocation.


Re: std.net.curl

2015-08-17 Thread anonymous via Digitalmars-d-learn

On Monday, 17 August 2015 at 13:13:48 UTC, anonymous wrote:


and figured out that the linker is invoked (on my machine) with
 gcc a.o -o a -m64 -lcurl -L/usr/lib/x86_64-linux-gnu -Xlinker


correct (I named the example programm a.d instead of app.d):
gcc app.o -o app -m64 -lcurl -L/usr/lib/x86_64-linux-gnu -Xlinker

This issue is already known:
 https://issues.dlang.org/show_bug.cgi?id=12572
and
 https://issues.dlang.org/show_bug.cgi?id=7044


Re: std.net.curl

2015-08-17 Thread tired_eyes via Digitalmars-d-learn

On Monday, 17 August 2015 at 13:26:29 UTC, Adam D. Ruppe wrote:

On Monday, 17 August 2015 at 13:04:31 UTC, tired_eyes wrote:

Error: unrecognized switch '-lcurl'


ooh I'm sorry, should have been `dmd -L-lcurl yourprogram.d`


Yes, it did the trick.



Re: std.net.curl

2015-08-17 Thread tired_eyes via Digitalmars-d-learn

On Monday, 17 August 2015 at 12:58:54 UTC, Adam D. Ruppe wrote:

On Monday, 17 August 2015 at 12:52:37 UTC, tired_eyes wrote:
Hovewer, dmd app.d spits a whole bunch of strange error 
messages:


try dmd -lcurl app.d and see if that helps.


Error: unrecognized switch '-lcurl'


Re: std.net.curl

2015-08-17 Thread Adam D. Ruppe via Digitalmars-d-learn

On Monday, 17 August 2015 at 12:52:37 UTC, tired_eyes wrote:
Hovewer, dmd app.d spits a whole bunch of strange error 
messages:


try dmd -lcurl app.d and see if that helps.


Re: std.net.curl

2015-08-17 Thread yawniek via Digitalmars-d-learn

On Monday, 17 August 2015 at 13:04:31 UTC, tired_eyes wrote:

On Monday, 17 August 2015 at 12:58:54 UTC, Adam D. Ruppe wrote:

On Monday, 17 August 2015 at 12:52:37 UTC, tired_eyes wrote:
Hovewer, dmd app.d spits a whole bunch of strange error 
messages:


try dmd -lcurl app.d and see if that helps.


Error: unrecognized switch '-lcurl'


do you use dub? if so, did you add
libs: [curl]
to your dub.json?

if that doesn't help, please post the output of curl-config.
e.g.
curl-config --libs --built-shared --prefix --static-libs



Re: std.net.curl

2015-08-17 Thread anonymous via Digitalmars-d-learn

On Monday, 17 August 2015 at 12:58:54 UTC, Adam D. Ruppe wrote:

On Monday, 17 August 2015 at 12:52:37 UTC, tired_eyes wrote:
Hovewer, dmd app.d spits a whole bunch of strange error 
messages:


try dmd -lcurl app.d and see if that helps.


DMD does not accept -lcurl. From dmd --help:
 -Llinkerflag   pass linkerflag to link
ergo
 dmd -L-lcurl app.d

on my machine (64Bit linux mint 17.2 with libcurl4-openssl-dev 
installed) -L-lcurl reduces the number of linker errors but the 
example still fails. I run

dmd -c app.d
dmd -L-lcurl app.o -v

and figured out that the linker is invoked (on my machine) with
 gcc a.o -o a -m64 -lcurl -L/usr/lib/x86_64-linux-gnu -Xlinker 
--export-dynamic -l:libphobos2.a -lpthread -lm -lrt
If I change the order of the libraries (move -lcurl to the end), 
it links. Used commands:

dmd -c app.d
gcc app.o -o app -m64  -L/usr/lib/x86_64-linux-gnu -Xlinker 
--export-dynamic -l:libphobos2.a -lpthread -lm -lrt -lcurl
(the later may be different on OpenSUSE (?) - better check with 
dmd -v ...)


Re: std.net.curl

2015-08-17 Thread Adam D. Ruppe via Digitalmars-d-learn

On Monday, 17 August 2015 at 13:04:31 UTC, tired_eyes wrote:

Error: unrecognized switch '-lcurl'


ooh I'm sorry, should have been `dmd -L-lcurl yourprogram.d`


Re: [std.net.curl] Downloading multiple files using download()

2014-12-28 Thread Jack via Digitalmars-d-learn
On Sunday, 28 December 2014 at 07:52:24 UTC, ketmar via 
Digitalmars-d-learn wrote:

On Sun, 28 Dec 2014 07:44:33 +
Jack via Digitalmars-d-learn 
digitalmars-d-learn@puremagic.com wrote:


How does one compile for Windows on a Linux Machine using dub? 
I've been using the platform : [windows] configuration in 
my dub.json to no avail since my Windows installation keeps on 
screaming out compatibility problems.

i don't think that you can do cross-developement with dub now. i
believe that the best thing you can do is to run dub.exe and 
dmd.exe
with wine. yet the last time i checked (that was a while ago) 
dmd.exe

with wine was able to produce only segfaults.

yet i'm not an expert in both dub and windows, so i don't 
really know.

i think that cross-developmenet question worth a separate topic.


Well that's a shame then. Thanks for the help ketmar.


Re: [std.net.curl] Downloading multiple files using download()

2014-12-27 Thread ketmar via Digitalmars-d-learn
On Sun, 28 Dec 2014 06:14:09 +
Jack via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote:

 I'm trying to create a sort of downloader where it will 
 download multiple pages of comics which are in .jpg format. 
 Now the problem is, that when I used the download() function:
 download(url, location);
 Where:
 url = direct link to the image
 location = /downloads/ ~ to!string(x) ~ .jpg; (x is the page 
 number)
 
 It would spew out an error code:
  std.stream.OpenException@std/stream.d(50): Cannot open or 
  create file '/downloads/1.jpg'
 
 I dug around to see that nothing but the documentation page has 
 an example for this, but unfortunately it was only for one file.
 
 So can anyone help me?
 
 Note: I can't get the url here since the url is a little bit 
 NSFW
you are trying to write the file to /download directory, that
obviously doesn't exist.

`url` arg is the FULL url -- the one that you'll pass to wget to
download the picture. and `location` is the DISK location of the
resulting file (i.e. the file that will be created). as you are passing
the absolute path for it, it starts searching from the root directory.

now remove that pesky '/' at the start of the `location` and give us
the link, as NSFW comics are specially good at sundays! ;-)


signature.asc
Description: PGP signature


Re: [std.net.curl] Downloading multiple files using download()

2014-12-27 Thread Jack via Digitalmars-d-learn
On Sunday, 28 December 2014 at 06:26:18 UTC, ketmar via 
Digitalmars-d-learn wrote:

On Sun, 28 Dec 2014 06:14:09 +
Jack via Digitalmars-d-learn 
digitalmars-d-learn@puremagic.com wrote:


I'm trying to create a sort of downloader where it will 
download multiple pages of comics which are in .jpg 
format. Now the problem is, that when I used the download() 
function:

download(url, location);
Where:
url = direct link to the image
location = /downloads/ ~ to!string(x) ~ .jpg; (x is the 
page number)


It would spew out an error code:
 std.stream.OpenException@std/stream.d(50): Cannot open or 
 create file '/downloads/1.jpg'


I dug around to see that nothing but the documentation page 
has an example for this, but unfortunately it was only for one 
file.


So can anyone help me?

Note: I can't get the url here since the url is a little bit 
NSFW

you are trying to write the file to /download directory, that
obviously doesn't exist.

`url` arg is the FULL url -- the one that you'll pass to wget to
download the picture. and `location` is the DISK location of the
resulting file (i.e. the file that will be created). as you are 
passing
the absolute path for it, it starts searching from the root 
directory.


now remove that pesky '/' at the start of the `location` and 
give us

the link, as NSFW comics are specially good at sundays! ;-)


As much as my brotherhood senses push me into revealing the 
links(they're in json), doing that will reveal my guilty 
pleasures and fetishes. Well, anywho, thanks for that. I 
appreciate it.


Re: [std.net.curl] Downloading multiple files using download()

2014-12-27 Thread ketmar via Digitalmars-d-learn
On Sun, 28 Dec 2014 06:51:14 +
Jack via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote:

 As much as my brotherhood senses push me into revealing the 
 links(they're in json), doing that will reveal my guilty 
 pleasures and fetishes. Well, anywho, thanks for that. I 
 appreciate it.
ah, 'cmon, nobody knows that you are you here! besides, you can sign as
Jack the Knife, for example. we will never realise that this is your
alter-ego!


signature.asc
Description: PGP signature


Re: [std.net.curl] Downloading multiple files using download()

2014-12-27 Thread Jack via Digitalmars-d-learn
On Sunday, 28 December 2014 at 06:59:02 UTC, ketmar via 
Digitalmars-d-learn wrote:

On Sun, 28 Dec 2014 06:51:14 +
Jack via Digitalmars-d-learn 
digitalmars-d-learn@puremagic.com wrote:


As much as my brotherhood senses push me into revealing the 
links(they're in json), doing that will reveal my guilty 
pleasures and fetishes. Well, anywho, thanks for that. I 
appreciate it.
ah, 'cmon, nobody knows that you are you here! besides, you can 
sign as
Jack the Knife, for example. we will never realise that this 
is your

alter-ego!


Well it's not really that much of a link really. I swear.
It's just like fapping to drawings or some sort of thing.
Not really THAT good of a comic. *sweats*


Re: [std.net.curl] Downloading multiple files using download()

2014-12-27 Thread ketmar via Digitalmars-d-learn
On Sun, 28 Dec 2014 07:24:53 +
Jack via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote:

 Well it's not really that much of a link really. I swear.
 It's just like fapping to drawings or some sort of thing.
 Not really THAT good of a comic. *sweats*
so that's not one of that stories where everyone dies right at the
first page? no napalm and burning mechas? ah, nevermind, that was one
of my perverted dreams...


signature.asc
Description: PGP signature


Re: [std.net.curl] Downloading multiple files using download()

2014-12-27 Thread Jack via Digitalmars-d-learn
On Sunday, 28 December 2014 at 07:33:23 UTC, ketmar via 
Digitalmars-d-learn wrote:

On Sun, 28 Dec 2014 07:24:53 +
Jack via Digitalmars-d-learn 
digitalmars-d-learn@puremagic.com wrote:



Well it's not really that much of a link really. I swear.
It's just like fapping to drawings or some sort of thing.
Not really THAT good of a comic. *sweats*
so that's not one of that stories where everyone dies right at 
the
first page? no napalm and burning mechas? ah, nevermind, that 
was one

of my perverted dreams...


Replace napalm with mayonaise and mechas with octopus legs.

Oh anyway, mind answering one more question?

How does one compile for Windows on a Linux Machine using dub? 
I've been using the platform : [windows] configuration in my 
dub.json to no avail since my Windows installation keeps on 
screaming out compatibility problems.


Re: [std.net.curl] Downloading multiple files using download()

2014-12-27 Thread ketmar via Digitalmars-d-learn
On Sun, 28 Dec 2014 07:44:33 +
Jack via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote:

 How does one compile for Windows on a Linux Machine using dub? 
 I've been using the platform : [windows] configuration in my 
 dub.json to no avail since my Windows installation keeps on 
 screaming out compatibility problems.
i don't think that you can do cross-developement with dub now. i
believe that the best thing you can do is to run dub.exe and dmd.exe
with wine. yet the last time i checked (that was a while ago) dmd.exe
with wine was able to produce only segfaults.

yet i'm not an expert in both dub and windows, so i don't really know.
i think that cross-developmenet question worth a separate topic.


signature.asc
Description: PGP signature


Re: std.net.curl - get() is too slow

2013-12-20 Thread Brad Anderson

On Friday, 20 December 2013 at 18:23:30 UTC, Benji wrote:

When I call get() function from std.net.curl,
I notice it's extremely slow!
Maybe 50 times slower than in Python..

Is there any better/faster alternative?


Without doing any profiling I'd say this character concatenation 
while decoding is probably a large source of any slowness.


https://github.com/D-Programming-Language/phobos/blob/master/std/net/curl.d#L1908

Switching it to Appender or doing some sort of batch processing 
would probably help a lot.  Even just a .reserve() would probably 
do wonders.


Re: std.net.curl - get() is too slow

2013-12-20 Thread David Nadlinger

On Friday, 20 December 2013 at 18:23:30 UTC, Benji wrote:

When I call get() function from std.net.curl,
I notice it's extremely slow!
Maybe 50 times slower than in Python..

Is there any better/faster alternative?


How do you benchmark the functions?

David


Re: std.net.curl on Windows

2013-08-09 Thread David
 Is there an easier way?  How to learn about it?  A detailed instruction
 (at least as detailed as the steps above) at the top of the manual page
 (http://dlang.org/phobos/std_net_curl.html) would have been nice...
 After all, it's a third party library not fully supplied with the compiler.

Thew installer has an option to install curl with D.



Re: std.net.curl on Windows

2013-08-09 Thread Ivan Kazmenko

On Friday, 9 August 2013 at 10:26:05 UTC, David wrote:
Is there an easier way?  How to learn about it?  A detailed 
instruction
(at least as detailed as the steps above) at the top of the 
manual page
(http://dlang.org/phobos/std_net_curl.html) would have been 
nice...
After all, it's a third party library not fully supplied with 
the compiler.


Thew installer has an option to install curl with D.


Thank you, that's good to know!  And the download page contains 
D-compatible libcurl download at the bottom, which I overlooked 
(albeit not the current version) - sorry!


Still, the all-platforms zip, which is the first download option 
(and so I happily grabbed it), seems to contain neither libcurl 
nor sufficient documentation on how to install it.  So my point 
that the docs are somewhat lacking is still valid.


Ivan Kazmenko.


Re: std.net.curl is not working?

2013-04-26 Thread qznc
Fri, 26 Apr 2013 19:25:16 +0200: mab wrote

 Why i get the following Error, when i try to compile a simple Hello
 World that imports std.net.curl=
 
 The Error:
 # dmd hello.d /usr/lib/x86_64-linux-gnu/libphobos2.a(curl.o): In
 function `_D3std3net4curl4Curl19_sharedStaticCtor28FZv':
 std/net/curl.d:(.text._D3std3net4curl4Curl19_sharedStaticCtor28FZv+0xf):
 undefined reference to `curl_global_init'
 /usr/lib/x86_64-linux-gnu/libphobos2.a(curl.o): In function
 `_D3std3net4curl4Curl19_sharedStaticDtor29FZv':
 std/net/curl.d:(.text._D3std3net4curl4Curl19_sharedStaticDtor29FZv+0x5):
 undefined reference to `curl_global_cleanup'
 collect2: ld returned 1 exit status --- errorlevel 1
 
 The Code:
 import std.stdio;
 import std.net.curl;
 
 void main()
 {
writeln(hello world);
 }
 
 My Testsystem: Debian # uname -a Linux dexplorer 2.6.32-5-amd64 #1 SMP
 Mon Feb 25 00:26:11 UTC 2013 x86_64 GNU/Linux
 
 curl is installed by apt-get install curl.
 
 Thanks!

You have to link libcurl via argument:

  dmd hello.d -L-lcurl


Re: std.net.curl is not working?

2013-04-26 Thread ollie
On Fri, 26 Apr 2013 19:25:16 +0200, mab wrote:

 undefined reference to `curl_global_init'
 undefined reference to `curl_global_cleanup'

These functions are defined in libcurl. Make sure you have installed 
libcurl if it wasn't installed as a dependency for curl.


Re: std.net.curl is not working?

2013-04-26 Thread mab

Thank you for answering. But it didnt work.

I get:
#dmd hello.d -L-lcurl
/usr/bin/ld: cannot find -lcurl
collect2: ld returned 1 exit status
--- errorlevel 1

Curl is installed, as also libcurl3.

I forget to mention that i am using DMD64 D Compiler v2.062.
Is std.net.curl working in this Version? Because it also didnt 
work on my Windows 7 System.


Re: std.net.curl is not working?

2013-04-26 Thread Jordi Sayol
On 26/04/13 19:55, mab wrote:
 Thank you for answering. But it didnt work.
 
 I get:
 #dmd hello.d -L-lcurl
 /usr/bin/ld: cannot find -lcurl
 collect2: ld returned 1 exit status
 --- errorlevel 1
 
 Curl is installed, as also libcurl3.

You need to install the development curl package:

$ sudo apt-get install libcurl4-openssl-dev
or
$ sudo apt-get install libcurl4-gnutls-dev
or
$ sudo apt-get install libcurl4-nss-dev

 
 I forget to mention that i am using DMD64 D Compiler v2.062.
 Is std.net.curl working in this Version? Because it also didnt work on my 
 Windows 7 System.
 

-- 
Jordi Sayol


Re: std.net.curl is not working?

2013-04-26 Thread Ali Çehreli

On 04/26/2013 10:55 AM, mab wrote:

 Thank you for answering. But it didnt work.

 I get:
 #dmd hello.d -L-lcurl
 /usr/bin/ld: cannot find -lcurl

Try providing the directory that the curl library file is in:

#dmd hello.d -L-L/wherever/libcurl/is/in -L-lcurl

Ali



Re: std.net.curl is not working?

2013-04-26 Thread John Colvin

On Friday, 26 April 2013 at 17:55:59 UTC, mab wrote:

Thank you for answering. But it didnt work.

I get:
#dmd hello.d -L-lcurl
/usr/bin/ld: cannot find -lcurl
collect2: ld returned 1 exit status
--- errorlevel 1

Curl is installed, as also libcurl3.

I forget to mention that i am using DMD64 D Compiler v2.062.
Is std.net.curl working in this Version? Because it also didnt 
work on my Windows 7 System.


Do you know what the libcurl library is actually called on your 
system? (try find /usr -name *curl*.so*  or find /usr -name 
*curl*.a* )


if the name you find is e.g. libcurl3.so then you'll have to 
use the linker flag -lcurl3 instead of lcurl


p.s. Please don't be offended if you're already fully aware of 
this, it's just better to make sure the basics are covered first.


Re: std.net.curl is not working?

2013-04-26 Thread mab

On Friday, 26 April 2013 at 18:14:04 UTC, Jordi Sayol wrote:
[...]

You need to install the development curl package:

$ sudo apt-get install libcurl4-openssl-dev
or
$ sudo apt-get install libcurl4-gnutls-dev
or
$ sudo apt-get install libcurl4-nss-dev


[...]

That´s it.
Thank you!


Re: std.net.curl - how to set custom Content-Type?

2012-09-18 Thread Johannes Pfau
Am Mon, 17 Sep 2012 22:35:39 +0200
schrieb Jonathan M Davis jmdavisp...@gmx.com:

 On Monday, September 17, 2012 20:59:05 Johannes Pfau wrote:
  addRequestHeader is quite dumb. It simply appends the header to a
  list. So by just calling it again you would actually send 2
  Content-Type headers.
 
 So, you're suggesting to send 2 content headers? That can't be good.
 It might work, but I'm pretty darn sure that it's against the HTTP
 spec to do so. You're only supposed to have duplicate headers when
 they're values are a list, and they can be concatenated into a single
 header.
 
 - Jonathan M Davis

No I'm not suggesting it, but that is what's being done if you call
addRequestHeader twice. Dmitry said addRequestHeader didn't work for
him and I wanted to explain that calling addRequestHeader again does
not overwrite the first value.


Re: std.net.curl - how to set custom Content-Type?

2012-09-18 Thread Dmitry Olshansky

On 18-Sep-12 11:41, Johannes Pfau wrote:

Am Mon, 17 Sep 2012 22:35:39 +0200
schrieb Jonathan M Davis jmdavisp...@gmx.com:


On Monday, September 17, 2012 20:59:05 Johannes Pfau wrote:

addRequestHeader is quite dumb. It simply appends the header to a
list. So by just calling it again you would actually send 2
Content-Type headers.


So, you're suggesting to send 2 content headers? That can't be good.
It might work, but I'm pretty darn sure that it's against the HTTP
spec to do so. You're only supposed to have duplicate headers when
they're values are a list, and they can be concatenated into a single
header.

- Jonathan M Davis


No I'm not suggesting it, but that is what's being done if you call
addRequestHeader twice. Dmitry said addRequestHeader didn't work for
him and I wanted to explain that calling addRequestHeader again does
not overwrite the first value.


Yeah, that's the case. Thanks Johannes, I'll use your workaround for now.

Still I believe it worths an enhancement request. Setting content type 
is a basic task.


--
Dmitry Olshansky


Re: std.net.curl - how to set custom Content-Type?

2012-09-17 Thread Johannes Pfau
Am Mon, 17 Sep 2012 22:33:28 +0400
schrieb Dmitry Olshansky dmitry.o...@gmail.com:

 Recently was playing around with std.net.curl high-level API.
 
 One thing that is a blocker for me is (quoting the docs):
 
 @property void postData(const(char)[] data);
 Specifying data to post when not using the onSend callback.
 ...
 Content-Type will default to text/plain. Data is not converted or 
 encoded by this method.
 
 Yeah, there are only 2 occurrences of Content-Type throughout the
 docs the second defaults to Content-Type application/octet-stream.
 
 Say I want to send text/xml. Adding Content-Type as header via
 addRequestHeader doesn't seem to change a thing (probably because it 
 already has the default one).
 
 
 

addRequestHeader is quite dumb. It simply appends the header to a list.
So by just calling it again you would actually send 2 Content-Type
headers.

Here's a short workaround: http://dpaste.dzfl.pl/4704965b


Re: std.net.curl - how to set custom Content-Type?

2012-09-17 Thread Jonathan M Davis
On Monday, September 17, 2012 20:59:05 Johannes Pfau wrote:
 addRequestHeader is quite dumb. It simply appends the header to a list.
 So by just calling it again you would actually send 2 Content-Type
 headers.

So, you're suggesting to send 2 content headers? That can't be good. It might 
work, but I'm pretty darn sure that it's against the HTTP spec to do so. 
You're only supposed to have duplicate headers when they're values are a list, 
and they can be concatenated into a single header.

- Jonathan M Davis


Re: std.net.curl get webpage asia font issue

2012-06-08 Thread Dmitry Olshansky

On 08.06.2012 5:03, Sam Hu wrote:

On Thursday, 7 June 2012 at 10:43:32 UTC, Dmitry Olshansky wrote:

string content = get(dlang.org);


It's simple this line you convert whatever site content was to
unicode. Problem is that convert is either broken or it's simply a
cast whereas it should re-encode source as unicode. So the way around
is to get it to array of bytes and decode yourself.



Thanks.May I know how ?Appreciated a piece of code segment.


seems like
ubyte[] data = get!(AutoProtocol, ubyte)(your-site.cn);
//should work, sorry I'm on windows and curl doesn't work here for me
then you work with your data, decode and whatever, at least this:
writeln(data);//will not throw but will print bytes

--
Dmitry Olshansky


Re: std.net.curl get webpage asia font issue

2012-06-07 Thread Kevin
On 07/06/12 02:57, Sam Hu wrote:
 string content = get(dlang.org);
 writefln(%s\n,content);

 So my very simple question is how to retrieve information from a
 webpage which could possibily contains asia font (like Chinese font)?

I'm not really sure but try:
wstring content = get(dlang.org);

Also make sure your terminal is set up for unicode.


Re: std.net.curl get webpage asia font issue

2012-06-07 Thread Dmitry Olshansky

On 07.06.2012 10:57, Sam Hu wrote:

Greeting!

The document on this website provide an example on how to get webpage
information by std.net.curl.It is quite straightforward:

[code]
import std.net.curl, std.stdio;

void main(){

// Return a string containing the content specified by an URL
string content = get(dlang.org);


It's simple this line you convert whatever site content was to 
unicode. Problem is that convert is either broken or it's simply a 
cast whereas it should re-encode source as unicode. So the way around is 
to get it to array of bytes and decode yourself.




writefln(%s\n,content);

readln;
}
[/code]

When I change get(dlang.org) to get(yahoo.com),everything goes
fine;but when I change to get(yahoo.com.cn),a runtime error said bad
gbk encoding bla...

So my very simple question is how to retrieve information from a webpage
which could possibily contains asia font (like Chinese font)?


I think it's not font but encoding problem.


Thanks for your help in advance.

Regards,
Sam



--
Dmitry Olshansky


Re: std.net.curl get webpage asia font issue

2012-06-07 Thread Sam Hu

On Thursday, 7 June 2012 at 10:43:32 UTC, Dmitry Olshansky wrote:

string content = get(dlang.org);


It's simple this line you convert whatever site content was 
to unicode. Problem is that convert is either broken or it's 
simply a cast whereas it should re-encode source as unicode. So 
the way around is to get it to array of bytes and decode 
yourself.




Thanks.May I know how ?Appreciated a piece of code segment.


Re: std.net.curl get webpage asia font issue

2012-06-07 Thread Sam Hu

On Thursday, 7 June 2012 at 10:38:53 UTC, Kevin wrote:

On 07/06/12 02:57, Sam Hu wrote:

string content = get(dlang.org);
writefln(%s\n,content);

So my very simple question is how to retrieve information from 
a
webpage which could possibily contains asia font (like Chinese 
font)?


I'm not really sure but try:
wstring content = get(dlang.org);

Also make sure your terminal is set up for unicode.


Sorry,no,it does not work,I tried to print the content to DFL
TextBox control but still the same issue.


Re: std.net.curl not working in 2.058 for Windows

2012-02-24 Thread Brad Anderson

On Friday, 24 February 2012 at 07:59:50 UTC, Brad Anderson wrote:
There is no documentation (both on dlang.org and in the local 
documentation) so I'm not sure if it's supposed to be working 
yet.  I get linker errors when I try to use it:


$ dmd netcurl.d
OPTLINK (R) for Win32  Release 8.00.12
Copyright (C) Digital Mars 1989-2010  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
netcurl.obj(netcurl)
 Error 42: Symbol Undefined 
_D3std3net4curl3FTP8__cpctorMxFKxS3std3net4curl3FTPZv

netcurl.obj(netcurl)
 Error 42: Symbol Undefined 
_D3std3net4curl3FTP11__fieldDtorMFZv

netcurl.obj(netcurl)
 Error 42: Symbol Undefined _D3std3net4curl3FTP7performMFZv
netcurl.obj(netcurl)
 Error 42: Symbol Undefined 
_D3std3net4curl13CurlException7__ClassZ

netcurl.obj(netcurl)
 Error 42: Symbol Undefined 
_D3std3net4curl13CurlException6__ctorMFAyaAyakC6object9ThrowableZC3std3net4curl13CurlException

netcurl.obj(netcurl)

[snip]

Furthermore the documentation in the actual source says this 
should work:


string content = get(http://dlang.org;);

But that results in a compiler error error:

netcurl.d(5): Error: cannot implicitly convert expression 
(get(http://dlang.org,AutoProtocol())) of type char[] to 
string


get() signature from the source is:

T[] get(Conn = AutoProtocol, T = char)(const(char)[] url, 
Conn conn = Conn())
if ( isCurlConn!Conn  (is(T == char) || is(T == 
ubyte)) )


An .idup fixes it, of course, but the documentation is wrong 
(or the signature is wrong).


Finally, was the curl library included?  If not, where can it 
be found?  It needs to be an OMF version, right?  I don't see 
an OMF version on curl's website.


Regards,
Brad Anderson



It would appear phobos's win32.mak wasn't updated to include the 
curl wrapper.


I can try to figure out how the makefile works and send a pull 
request if nobody who knows what's going on has time.


Regards,
Brad Anderson


Re: std.net.curl not working in 2.058 for Windows

2012-02-24 Thread Brad Anderson

On Friday, 24 February 2012 at 08:47:14 UTC, Brad Anderson wrote:
On Friday, 24 February 2012 at 07:59:50 UTC, Brad Anderson 
wrote:
There is no documentation (both on dlang.org and in the local 
documentation) so I'm not sure if it's supposed to be working 
yet.  I get linker errors when I try to use it:


   $ dmd netcurl.d
   OPTLINK (R) for Win32  Release 8.00.12
   Copyright (C) Digital Mars 1989-2010  All rights reserved.
   http://www.digitalmars.com/ctg/optlink.html
   netcurl.obj(netcurl)
Error 42: Symbol Undefined 
_D3std3net4curl3FTP8__cpctorMxFKxS3std3net4curl3FTPZv

   netcurl.obj(netcurl)
Error 42: Symbol Undefined 
_D3std3net4curl3FTP11__fieldDtorMFZv

   netcurl.obj(netcurl)
Error 42: Symbol Undefined _D3std3net4curl3FTP7performMFZv
   netcurl.obj(netcurl)
Error 42: Symbol Undefined 
_D3std3net4curl13CurlException7__ClassZ

   netcurl.obj(netcurl)
Error 42: Symbol Undefined 
_D3std3net4curl13CurlException6__ctorMFAyaAyakC6object9ThrowableZC3std3net4curl13CurlException

   netcurl.obj(netcurl)

   [snip]

Furthermore the documentation in the actual source says this 
should work:


   string content = get(http://dlang.org;);

But that results in a compiler error error:

   netcurl.d(5): Error: cannot implicitly convert expression 
(get(http://dlang.org,AutoProtocol())) of type char[] to 
string


get() signature from the source is:

   T[] get(Conn = AutoProtocol, T = char)(const(char)[] url, 
Conn conn = Conn())
   if ( isCurlConn!Conn  (is(T == char) || is(T == 
ubyte)) )


An .idup fixes it, of course, but the documentation is wrong 
(or the signature is wrong).


Finally, was the curl library included?  If not, where can it 
be found?  It needs to be an OMF version, right?  I don't see 
an OMF version on curl's website.


Regards,
Brad Anderson



It would appear phobos's win32.mak wasn't updated to include 
the curl wrapper.


I can try to figure out how the makefile works and send a pull 
request if nobody who knows what's going on has time.


Regards,
Brad Anderson


https://github.com/D-Programming-Language/phobos/pull/458
https://github.com/D-Programming-Language/d-programming-language.org/pull/91

Regards,
Brad Anderson