Re: HTTP-methods and encoding
On Saturday, 7 April 2018 at 23:54:21 UTC, Vindex wrote:
On Saturday, 7 April 2018 at 15:58:14 UTC, Seb wrote:
On Saturday, 7 April 2018 at 13:02:39 UTC, Vindex wrote:
There is an error on some sites when using HTTP-methods
(std.net.curl.get, std.net.curl.post):
std.encoding.EncodingException@std/encoding.d(2505):
Unrecognized Encoding: utf8
Is there a beautiful way around it?
For the GET-method I use the download() and readText(). But
for the POST-method I can not come up with an alternative
solution.
That's weird. std.net.curl should be able to handle UTF-8.
What content are you trying to download/post?
Maybe you can open a bug report for it?
In any case, you might want to checkout requests:
https://github.com/ikod/dlang-requests
It's by far more convenient to use than std.net.curl
The problem is that "utf-8" (or "UTF-8") is required instead of
"utf8".
I tried to get HTML and JSON. For example, this GET-query
returns error:
"https://yobit.net/api/3/ticker/btc_usd";
Thank you for the advice
Hello,
"utf-8" (or "UTF-8") is required instead of "utf8" - explain,
please.
Anyway this code works as expected:
import requests;
import std.stdio;
import std.format;
void main() {
auto rq = Request();
auto rs = rq.get("https://yobit.net/api/3/ticker/btc_usd";);
writeln(rs.responseBody);
}
output:
{"btc_usd":{"high":7216.09463851,"low":6950,"avg":7083.04731925,"vol":753989.73116823,"vol_cur":105.94453165,"last":7114,"buy":7114.,"sell":7135.1911,"updated":1523170067}}
Re: Benchmarking sigmoid function between C and D
On Saturday, 7 April 2018 at 23:48:36 UTC, kinke wrote: On Saturday, 7 April 2018 at 20:33:13 UTC, Arun Chandrasekaran wrote: [...] As this appears to be benchmarking mostly the std.math.exp(float) performance - some/many basic algos in std.math, incl. exp(), are currently using the x87 FPU for all 3 floating-point types, although there are treacherous float/double overloads, sacrificing performance in many cases. See https://github.com/dlang/phobos/pull/6272#issuecomment-373967109 (and the later numbers for current Intel and GCC compilers) for a performance comparison of some std.math algos incl. exp(). For LDC, the double/float versions of the few worked-on algos in that PR were sped-up by an overall factor of 3 [but only by 1.46 for exp(float)]. Interesting to see this, thanks! Did you also generate the bar graph plot using D?
Re: HTTP-methods and encoding
On Saturday, 7 April 2018 at 15:58:14 UTC, Seb wrote: On Saturday, 7 April 2018 at 13:02:39 UTC, Vindex wrote: There is an error on some sites when using HTTP-methods (std.net.curl.get, std.net.curl.post): std.encoding.EncodingException@std/encoding.d(2505): Unrecognized Encoding: utf8 Is there a beautiful way around it? For the GET-method I use the download() and readText(). But for the POST-method I can not come up with an alternative solution. That's weird. std.net.curl should be able to handle UTF-8. What content are you trying to download/post? Maybe you can open a bug report for it? In any case, you might want to checkout requests: https://github.com/ikod/dlang-requests It's by far more convenient to use than std.net.curl The problem is that "utf-8" (or "UTF-8") is required instead of "utf8". I tried to get HTML and JSON. For example, this GET-query returns error: "https://yobit.net/api/3/ticker/btc_usd"; Thank you for the advice
Re: Benchmarking sigmoid function between C and D
On Saturday, 7 April 2018 at 20:33:13 UTC, Arun Chandrasekaran wrote: Much better with mir.math.common, still a bit slower than C (even with larger loops): As this appears to be benchmarking mostly the std.math.exp(float) performance - some/many basic algos in std.math, incl. exp(), are currently using the x87 FPU for all 3 floating-point types, although there are treacherous float/double overloads, sacrificing performance in many cases. See https://github.com/dlang/phobos/pull/6272#issuecomment-373967109 (and the later numbers for current Intel and GCC compilers) for a performance comparison of some std.math algos incl. exp(). For LDC, the double/float versions of the few worked-on algos in that PR were sped-up by an overall factor of 3 [but only by 1.46 for exp(float)].
Re: Benchmarking sigmoid function between C and D
On Saturday, 7 April 2018 at 20:33:13 UTC, Arun Chandrasekaran wrote: On Saturday, 7 April 2018 at 19:14:27 UTC, Daniel Kozak wrote: or for ldc http://docs.algorithm.dlang.io/latest/mir_math_common.html On Sat, Apr 7, 2018 at 9:10 PM, Daniel Kozak wrote: can you try it with c math functions? instead of std.math, try to use core.stdc.math On Sat, Apr 7, 2018 at 8:53 PM, Arun Chandrasekaran via Digitalmars-d-learn wrote: [...] Much better with mir.math.common, still a bit slower than C (even with larger loops): 10^7 iterations using sigmoid1: 168 ms 10^7 iterations using sigmoid2: 39 ms Also LDC optimized away the computation. So I had to modify the code a bit. Have you tried LLVM intrinsics? say llvm_exp
Re: Store struct tuple of alias and access members through it?
On Saturday, 7 April 2018 at 13:31:01 UTC, Timoses wrote:
Simen was faster :)
In my solution I simply ignore such things as functions... But
there is the cool delegate creation approach in Simen's solution
for this. I can handle arrays instead. :)
And I got rid of tupelof acting on an instance.
Be aware, that bitfields create more fields then the delegates
for the bitfield's members...
import std.stdio;
import std.bitmanip;
import std.traits;
void main()
{
S s;
Param!S example = new Param!S(s);
writeln(example[0]);
writeln(example[1]);
writeln(example[2]);
writeln(example[3]);
writeln(example[4]);
writeln(example[5]);
writeln(example[6]);
writeln([__traits(allMembers, S)]);
writeln(example[15]);
writeln(example[16]);
writeln(example[17]);
writeln(example[18]);
}
struct S
{
uint s1;
ushort s2;
string s3;
float s4;
mixin(bitfields!(
uint, "x",2,
int, "y",3,
uint, "z",2,
bool, "flag", 1));
size_t fun(){ return 42; }
size_t delegate() dg;
size_t[] arr;
static void fun(){}
}
interface IParam{}
class Param(T) : IParam
{
T m;
this(T m)
{
this.m = m;
}
IParam opIndex(size_t i)
{
//static if(!isBasicType!T)
static if(__traits(compiles, __traits(allMembers, T)))
{
static foreach (j, t; __traits(allMembers, T))
{
if (i == j)
{
static if(__traits(compiles, new
Param!(typeof(__traits(getMember, this.m,
t)))(__traits(getMember, this.m, t
{
return new Param!(typeof(__traits(getMember,
this.m, t)))(__traits(getMember, this.m, t));
}
}
}
}
return null;
}
}
Re: Benchmarking sigmoid function between C and D
On Saturday, 7 April 2018 at 19:14:27 UTC, Daniel Kozak wrote:
or for ldc
http://docs.algorithm.dlang.io/latest/mir_math_common.html
On Sat, Apr 7, 2018 at 9:10 PM, Daniel Kozak
wrote:
can you try it with c math functions?
instead of std.math, try to use core.stdc.math
On Sat, Apr 7, 2018 at 8:53 PM, Arun Chandrasekaran via
Digitalmars-d-learn wrote:
[...]
Much better with mir.math.common, still a bit slower than C (even
with larger loops):
10^7 iterations using sigmoid1: 168 ms
10^7 iterations using sigmoid2: 39 ms
Also LDC optimized away the computation. So I had to modify the
code a bit.
```
private auto sigmoid1_perf() {
auto sw = StopWatch(AutoStart.yes);
int i;
float x, y = 0.0f;
for (i = 0; i < 10; i++) {
for (x = -5.0f; x <= 5.0f; x+=0.1f) {
y += sigmoid1(x);
}
}
auto t = sw.peek.total!"msecs";
return tuple(y, t);
}
private auto sigmoid2_perf() {
auto sw = StopWatch(AutoStart.yes);
int i;
float x, y = 0.0f;
for (i = 0; i < 10; i++) {
for (x = -5.0f; x <= 5.0f; x+=0.1f) {
y += sigmoid2(x);
}
}
auto t = sw.peek.total!"msecs";
return tuple(y, t);
}
```
Re: Store struct tuple of alias and access members through it?
On Saturday, 7 April 2018 at 13:31:01 UTC, Timoses wrote:
In the end I would like to accomplish the following:
Provide access to contained bitfields and members of a struct
in the order they
appear in the struct via an index.
The behavior of Type.tupleof in D seems a bit unfinished - they
can't be passed to other functions, they can't be directly used
to get the member they refer to, and the indirect way is clunky.
Anyways. Your desired code `return this.m.members[i];` is, as you
have noticed, impossible. There's multiple reasons for that -
first, `members` can't be used like that. Second, since you need
to wrap it in a Param instance, you need more information than
that. Third, there's a clear distinction in D between
compile-time and run-time values, so you need the static foreach
there to get the right compile-time value.
Now, what *can* we do?
First, there's no need for __traits(getMember, this.m,
members[j].stringof), since the index into T.tupleof is the exact
same as for m.tupleof. In other words, you could use
return new Param!(typeof(m.tupleof[j]))(m.tupleof[j]);
I think that is clearer. I'd suggest also creating this function:
IParam param(T)(T value) {
return new Param!T(value);
}
That way, the above line would be
return param(m.tupleof[j]);
Handling methods is a tad more complicated, and you will not get
the correct interleaving of methods and fields, which may or may
not be a problem to you.
Here's my attempt at solving all your problems. There may be
things I've misunderstood, forgotten or ignored, and there are
certainly places where I'm unsure.
import std.meta;
import std.traits;
// List all member functions, and wrap them such that
myFoo.fun(3) can be called as
AllMemberFunctions!(typeof(myFoo))[idx](myFoo, 3).
template AllMemberFunctions(T)
{
template createDg(alias fn)
{
static if (__traits(isStaticFunction, fn))
alias createDg = fn;
else
ReturnType!fn createDg(ref T ctx, Parameters!fn args)
{
ReturnType!fn delegate(Parameters!fn) fun;
fun.funcptr = &fn;
fun.ptr = cast(void*)&ctx;
return fun(args);
}
}
alias GetOverloads(string name) =
AliasSeq!(__traits(getOverloads, T, name));
alias AllMemberFunctions = staticMap!(createDg,
staticMap!(GetOverloads, __traits(allMembers, T)));
}
interface IParam
{
// Moved this here, since otherwise you'd need to know the
// exact template parameters to Param to get to anything.
IParam opIndex(size_t i);
}
// Simplified template definition.
class Param(T) : IParam
{
T m;
this(T m)
{
this.m = m;
}
static if (!isBasicType!T && !isArray!T &&
!isFunctionPointer!T)
{
IParam opIndex(size_t i)
{
switch (i)
{
// Go through all members:
static foreach (j; 0..m.tupleof.length)
case j:
return param(m.tupleof[j]);
// Then all functions after:
static foreach (j, fn; AllMemberFunctions!T)
case j+m.tupleof.length:
return param(&fn);
// And blow up if the index is invalid.
default:
assert(false, "Invalid index!");
}
}
}
else
{
IParam opIndex(size_t i)
{
assert(false, T.stringof ~ " is not an aggregate
type, and can't be indexed.");
}
}
}
IParam param(T)(T value) {
return new Param!T(value);
}
struct S {
int n;
float f;
string s;
int fn() { return n+2; }
string fun() { return ""; }
string fun(int n) { return ""; }
static void func() {}
}
unittest {
S s;
IParam a = param(s);
}
--
Simen
Re: Benchmarking sigmoid function between C and D
On Saturday, 7 April 2018 at 18:53:57 UTC, Arun Chandrasekaran wrote: What am I doing wrong here that makes the D equivalent 2.5 times slower than it's C equivalent? Compilers used: LDC2: LDC - the LLVM D compiler (1.8.0) GCC: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 11:36:39 ~/code/c/test2$ ldc2 sigmoid.d -O5 && ./sigmoid When benchmarking against C you need to add more switches to ldc2, like: -release -boundscheck=off
Re: Benchmarking sigmoid function between C and D
or for ldc http://docs.algorithm.dlang.io/latest/mir_math_common.html
On Sat, Apr 7, 2018 at 9:10 PM, Daniel Kozak wrote:
> can you try it with c math functions?
>
> instead of std.math, try to use core.stdc.math
>
> On Sat, Apr 7, 2018 at 8:53 PM, Arun Chandrasekaran via
> Digitalmars-d-learn wrote:
>
>> What am I doing wrong here that makes the D equivalent 2.5 times slower
>> than it's C equivalent?
>>
>> Compilers used:
>>
>> LDC2: LDC - the LLVM D compiler (1.8.0)
>> GCC: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
>>
>> 11:36:39 ~/code/c/test2$ ldc2 sigmoid.d -O5 && ./sigmoid
>> Max deviation is 0.001664
>> 10^7 iterations using sigmoid1: 308 ms
>> 10^7 iterations using sigmoid2: 30 ms
>> 11:36:55 ~/code/c/test2
>> $ gcc sigmoid.c -o sigmoid-c -O3 -lm 2>/dev/null && ./sigmoid-c
>> Max deviation is 0.001664
>> 10^7 iterations using sigmoid1: 134 ms
>> 10^7 iterations using sigmoid2: 29 ms
>> 11:37:10 ~/code/c/test2
>> $
>>
>> C code, taken from https://stackoverflow.com/ques
>> tions/412019/math-optimization-in-c-sharp#412176:
>>
>> ```
>> #include
>> #include
>> #include
>>
>> #define SCALE 320.0f
>> #define RESOLUTION 2047
>> #define MIN -RESOLUTION / SCALE
>> #define MAX RESOLUTION / SCALE
>>
>> static float sigmoid_lut[RESOLUTION + 1];
>>
>> void init_sigmoid_lut(void) {
>> int i;
>> for (i = 0; i < RESOLUTION + 1; i++) {
>> sigmoid_lut[i] = (1.0 / (1.0 + exp(-i / SCALE)));
>> }
>> }
>>
>> static float sigmoid1(const float value) {
>> return (1.0f / (1.0f + expf(-value)));
>> }
>>
>> static float sigmoid2(const float value) {
>> if (value <= MIN) return 0.0f;
>> if (value >= MAX) return 1.0f;
>> if (value >= 0) return sigmoid_lut[(int)(value * SCALE + 0.5f)];
>> return 1.0f-sigmoid_lut[(int)(-value * SCALE + 0.5f)];
>> }
>>
>> float test_error() {
>> float x;
>> float emax = 0.0;
>>
>> for (x = -10.0f; x < 10.0f; x+=0.1f) {
>> float v0 = sigmoid1(x);
>> float v1 = sigmoid2(x);
>> float error = fabsf(v1 - v0);
>> if (error > emax) { emax = error; }
>> }
>> return emax;
>> }
>>
>> int sigmoid1_perf() {
>> clock_t t0, t1;
>> int i;
>> float x, y = 0.0f;
>>
>> t0 = clock();
>> for (i = 0; i < 10; i++) {
>> for (x = -5.0f; x <= 5.0f; x+=0.1f) {
>> y = sigmoid1(x);
>> }
>> }
>> t1 = clock();
>> printf("", y); /* To avoid sigmoidX() calls being optimized away */
>> return (t1 - t0) / (CLOCKS_PER_SEC / 1000);
>> }
>>
>> int sigmoid2_perf() {
>> clock_t t0, t1;
>> int i;
>> float x, y = 0.0f;
>> t0 = clock();
>> for (i = 0; i < 10; i++) {
>> for (x = -5.0f; x <= 5.0f; x+=0.1f) {
>> y = sigmoid2(x);
>> }
>> }
>> t1 = clock();
>> printf("", y); /* To avoid sigmoidX() calls being optimized away */
>> return (t1 - t0) / (CLOCKS_PER_SEC / 1000);
>> }
>>
>> int main(void) {
>> init_sigmoid_lut();
>> printf("Max deviation is %0.6f\n", test_error());
>> printf("10^7 iterations using sigmoid1: %d ms\n", sigmoid1_perf());
>> printf("10^7 iterations using sigmoid2: %d ms\n", sigmoid2_perf());
>>
>> return 0;
>> }
>> ```
>>
>> D equivalent:
>>
>> ```
>> module sigmoid;
>>
>> import std.stdio;
>> import std.math;
>> import std.datetime.stopwatch;
>>
>> enum SCALE = 320.0f;
>> enum RESOLUTION = 2047;
>> enum MIN = -RESOLUTION / SCALE;
>> enum MAX = RESOLUTION / SCALE;
>>
>> float[RESOLUTION + 1] sigmoid_lut;
>>
>> void init_sigmoid_lut() {
>> int i;
>> for (i = 0; i < RESOLUTION + 1; i++) {
>> sigmoid_lut[i] = (1.0 / (1.0 + exp(-i / SCALE)));
>> }
>> }
>>
>> private float sigmoid1(const float value) {
>> return (1.0f / (1.0f + exp(-value)));
>> }
>>
>> private float sigmoid2(const float value) {
>> if (value <= MIN) return 0.0f;
>> if (value >= MAX) return 1.0f;
>> if (value >= 0) return sigmoid_lut[cast(int)(value * SCALE + 0.5f)];
>> return 1.0f-sigmoid_lut[cast(int)(-value * SCALE + 0.5f)];
>> }
>>
>> private float test_error() {
>> float x;
>> float emax = 0.0;
>>
>> for (x = -10.0f; x < 10.0f; x+=0.1f) {
>> float v0 = sigmoid1(x);
>> float v1 = sigmoid2(x);
>> float error = fabs(v1 - v0);
>> if (error > emax) { emax = error; }
>> }
>> return emax;
>> }
>>
>> private auto sigmoid1_perf() {
>> auto sw = StopWatch(AutoStart.yes);
>> int i;
>> float x, y = 0.0f;
>>
>> for (i = 0; i < 10; i++) {
>> for (x = -5.0f; x <= 5.0f; x+=0.1f) {
>> y = sigmoid1(x);
>> }
>> }
>> return sw.peek.total!"msecs";
>> }
>>
>> private auto sigmoid2_perf() {
>> auto sw = StopWatch(AutoStart.yes);
>> int i;
>> float x, y = 0.0f;
>> for (i = 0; i < 10; i++) {
>> for (x = -5.0f; x <= 5.0f; x+=0.1f) {
>> y = sigmoid2(x);
>> }
>> }
>> return sw.peek.total!"msecs";
>> }
>>
>> in
Re: Benchmarking sigmoid function between C and D
can you try it with c math functions?
instead of std.math, try to use core.stdc.math
On Sat, Apr 7, 2018 at 8:53 PM, Arun Chandrasekaran via Digitalmars-d-learn
wrote:
> What am I doing wrong here that makes the D equivalent 2.5 times slower
> than it's C equivalent?
>
> Compilers used:
>
> LDC2: LDC - the LLVM D compiler (1.8.0)
> GCC: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
>
> 11:36:39 ~/code/c/test2$ ldc2 sigmoid.d -O5 && ./sigmoid
> Max deviation is 0.001664
> 10^7 iterations using sigmoid1: 308 ms
> 10^7 iterations using sigmoid2: 30 ms
> 11:36:55 ~/code/c/test2
> $ gcc sigmoid.c -o sigmoid-c -O3 -lm 2>/dev/null && ./sigmoid-c
> Max deviation is 0.001664
> 10^7 iterations using sigmoid1: 134 ms
> 10^7 iterations using sigmoid2: 29 ms
> 11:37:10 ~/code/c/test2
> $
>
> C code, taken from https://stackoverflow.com/ques
> tions/412019/math-optimization-in-c-sharp#412176:
>
> ```
> #include
> #include
> #include
>
> #define SCALE 320.0f
> #define RESOLUTION 2047
> #define MIN -RESOLUTION / SCALE
> #define MAX RESOLUTION / SCALE
>
> static float sigmoid_lut[RESOLUTION + 1];
>
> void init_sigmoid_lut(void) {
> int i;
> for (i = 0; i < RESOLUTION + 1; i++) {
> sigmoid_lut[i] = (1.0 / (1.0 + exp(-i / SCALE)));
> }
> }
>
> static float sigmoid1(const float value) {
> return (1.0f / (1.0f + expf(-value)));
> }
>
> static float sigmoid2(const float value) {
> if (value <= MIN) return 0.0f;
> if (value >= MAX) return 1.0f;
> if (value >= 0) return sigmoid_lut[(int)(value * SCALE + 0.5f)];
> return 1.0f-sigmoid_lut[(int)(-value * SCALE + 0.5f)];
> }
>
> float test_error() {
> float x;
> float emax = 0.0;
>
> for (x = -10.0f; x < 10.0f; x+=0.1f) {
> float v0 = sigmoid1(x);
> float v1 = sigmoid2(x);
> float error = fabsf(v1 - v0);
> if (error > emax) { emax = error; }
> }
> return emax;
> }
>
> int sigmoid1_perf() {
> clock_t t0, t1;
> int i;
> float x, y = 0.0f;
>
> t0 = clock();
> for (i = 0; i < 10; i++) {
> for (x = -5.0f; x <= 5.0f; x+=0.1f) {
> y = sigmoid1(x);
> }
> }
> t1 = clock();
> printf("", y); /* To avoid sigmoidX() calls being optimized away */
> return (t1 - t0) / (CLOCKS_PER_SEC / 1000);
> }
>
> int sigmoid2_perf() {
> clock_t t0, t1;
> int i;
> float x, y = 0.0f;
> t0 = clock();
> for (i = 0; i < 10; i++) {
> for (x = -5.0f; x <= 5.0f; x+=0.1f) {
> y = sigmoid2(x);
> }
> }
> t1 = clock();
> printf("", y); /* To avoid sigmoidX() calls being optimized away */
> return (t1 - t0) / (CLOCKS_PER_SEC / 1000);
> }
>
> int main(void) {
> init_sigmoid_lut();
> printf("Max deviation is %0.6f\n", test_error());
> printf("10^7 iterations using sigmoid1: %d ms\n", sigmoid1_perf());
> printf("10^7 iterations using sigmoid2: %d ms\n", sigmoid2_perf());
>
> return 0;
> }
> ```
>
> D equivalent:
>
> ```
> module sigmoid;
>
> import std.stdio;
> import std.math;
> import std.datetime.stopwatch;
>
> enum SCALE = 320.0f;
> enum RESOLUTION = 2047;
> enum MIN = -RESOLUTION / SCALE;
> enum MAX = RESOLUTION / SCALE;
>
> float[RESOLUTION + 1] sigmoid_lut;
>
> void init_sigmoid_lut() {
> int i;
> for (i = 0; i < RESOLUTION + 1; i++) {
> sigmoid_lut[i] = (1.0 / (1.0 + exp(-i / SCALE)));
> }
> }
>
> private float sigmoid1(const float value) {
> return (1.0f / (1.0f + exp(-value)));
> }
>
> private float sigmoid2(const float value) {
> if (value <= MIN) return 0.0f;
> if (value >= MAX) return 1.0f;
> if (value >= 0) return sigmoid_lut[cast(int)(value * SCALE + 0.5f)];
> return 1.0f-sigmoid_lut[cast(int)(-value * SCALE + 0.5f)];
> }
>
> private float test_error() {
> float x;
> float emax = 0.0;
>
> for (x = -10.0f; x < 10.0f; x+=0.1f) {
> float v0 = sigmoid1(x);
> float v1 = sigmoid2(x);
> float error = fabs(v1 - v0);
> if (error > emax) { emax = error; }
> }
> return emax;
> }
>
> private auto sigmoid1_perf() {
> auto sw = StopWatch(AutoStart.yes);
> int i;
> float x, y = 0.0f;
>
> for (i = 0; i < 10; i++) {
> for (x = -5.0f; x <= 5.0f; x+=0.1f) {
> y = sigmoid1(x);
> }
> }
> return sw.peek.total!"msecs";
> }
>
> private auto sigmoid2_perf() {
> auto sw = StopWatch(AutoStart.yes);
> int i;
> float x, y = 0.0f;
> for (i = 0; i < 10; i++) {
> for (x = -5.0f; x <= 5.0f; x+=0.1f) {
> y = sigmoid2(x);
> }
> }
> return sw.peek.total!"msecs";
> }
>
> int main() {
> init_sigmoid_lut();
> writefln("Max deviation is %0.6f", test_error());
> writefln("10^7 iterations using sigmoid1: %s ms", sigmoid1_perf());
> writefln("10^7 iterations using sigmoid2: %s ms", sigmoid2_perf());
>
> return 0;
> }
> ```
>
Benchmarking sigmoid function between C and D
What am I doing wrong here that makes the D equivalent 2.5 times
slower than it's C equivalent?
Compilers used:
LDC2: LDC - the LLVM D compiler (1.8.0)
GCC: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
11:36:39 ~/code/c/test2$ ldc2 sigmoid.d -O5 && ./sigmoid
Max deviation is 0.001664
10^7 iterations using sigmoid1: 308 ms
10^7 iterations using sigmoid2: 30 ms
11:36:55 ~/code/c/test2
$ gcc sigmoid.c -o sigmoid-c -O3 -lm 2>/dev/null && ./sigmoid-c
Max deviation is 0.001664
10^7 iterations using sigmoid1: 134 ms
10^7 iterations using sigmoid2: 29 ms
11:37:10 ~/code/c/test2
$
C code, taken from
https://stackoverflow.com/questions/412019/math-optimization-in-c-sharp#412176:
```
#include
#include
#include
#define SCALE 320.0f
#define RESOLUTION 2047
#define MIN -RESOLUTION / SCALE
#define MAX RESOLUTION / SCALE
static float sigmoid_lut[RESOLUTION + 1];
void init_sigmoid_lut(void) {
int i;
for (i = 0; i < RESOLUTION + 1; i++) {
sigmoid_lut[i] = (1.0 / (1.0 + exp(-i / SCALE)));
}
}
static float sigmoid1(const float value) {
return (1.0f / (1.0f + expf(-value)));
}
static float sigmoid2(const float value) {
if (value <= MIN) return 0.0f;
if (value >= MAX) return 1.0f;
if (value >= 0) return sigmoid_lut[(int)(value * SCALE +
0.5f)];
return 1.0f-sigmoid_lut[(int)(-value * SCALE + 0.5f)];
}
float test_error() {
float x;
float emax = 0.0;
for (x = -10.0f; x < 10.0f; x+=0.1f) {
float v0 = sigmoid1(x);
float v1 = sigmoid2(x);
float error = fabsf(v1 - v0);
if (error > emax) { emax = error; }
}
return emax;
}
int sigmoid1_perf() {
clock_t t0, t1;
int i;
float x, y = 0.0f;
t0 = clock();
for (i = 0; i < 10; i++) {
for (x = -5.0f; x <= 5.0f; x+=0.1f) {
y = sigmoid1(x);
}
}
t1 = clock();
printf("", y); /* To avoid sigmoidX() calls being optimized
away */
return (t1 - t0) / (CLOCKS_PER_SEC / 1000);
}
int sigmoid2_perf() {
clock_t t0, t1;
int i;
float x, y = 0.0f;
t0 = clock();
for (i = 0; i < 10; i++) {
for (x = -5.0f; x <= 5.0f; x+=0.1f) {
y = sigmoid2(x);
}
}
t1 = clock();
printf("", y); /* To avoid sigmoidX() calls being optimized
away */
return (t1 - t0) / (CLOCKS_PER_SEC / 1000);
}
int main(void) {
init_sigmoid_lut();
printf("Max deviation is %0.6f\n", test_error());
printf("10^7 iterations using sigmoid1: %d ms\n",
sigmoid1_perf());
printf("10^7 iterations using sigmoid2: %d ms\n",
sigmoid2_perf());
return 0;
}
```
D equivalent:
```
module sigmoid;
import std.stdio;
import std.math;
import std.datetime.stopwatch;
enum SCALE = 320.0f;
enum RESOLUTION = 2047;
enum MIN = -RESOLUTION / SCALE;
enum MAX = RESOLUTION / SCALE;
float[RESOLUTION + 1] sigmoid_lut;
void init_sigmoid_lut() {
int i;
for (i = 0; i < RESOLUTION + 1; i++) {
sigmoid_lut[i] = (1.0 / (1.0 + exp(-i / SCALE)));
}
}
private float sigmoid1(const float value) {
return (1.0f / (1.0f + exp(-value)));
}
private float sigmoid2(const float value) {
if (value <= MIN) return 0.0f;
if (value >= MAX) return 1.0f;
if (value >= 0) return sigmoid_lut[cast(int)(value * SCALE +
0.5f)];
return 1.0f-sigmoid_lut[cast(int)(-value * SCALE + 0.5f)];
}
private float test_error() {
float x;
float emax = 0.0;
for (x = -10.0f; x < 10.0f; x+=0.1f) {
float v0 = sigmoid1(x);
float v1 = sigmoid2(x);
float error = fabs(v1 - v0);
if (error > emax) { emax = error; }
}
return emax;
}
private auto sigmoid1_perf() {
auto sw = StopWatch(AutoStart.yes);
int i;
float x, y = 0.0f;
for (i = 0; i < 10; i++) {
for (x = -5.0f; x <= 5.0f; x+=0.1f) {
y = sigmoid1(x);
}
}
return sw.peek.total!"msecs";
}
private auto sigmoid2_perf() {
auto sw = StopWatch(AutoStart.yes);
int i;
float x, y = 0.0f;
for (i = 0; i < 10; i++) {
for (x = -5.0f; x <= 5.0f; x+=0.1f) {
y = sigmoid2(x);
}
}
return sw.peek.total!"msecs";
}
int main() {
init_sigmoid_lut();
writefln("Max deviation is %0.6f", test_error());
writefln("10^7 iterations using sigmoid1: %s ms",
sigmoid1_perf());
writefln("10^7 iterations using sigmoid2: %s ms",
sigmoid2_perf());
return 0;
}
```
Re: std.datetime.systime: days Deprecation message
On Saturday, 7 April 2018 at 17:25:07 UTC, Vino wrote:
Hi All,
Request your help on the below Deprecation message.
import std.datetime.systime: Clock, days, SysTime;
void main (int AgeSize) {
int AgeSize = 1
auto ct2 = Clock.currTime(), st2 = ct2 + days(-AgeSize);
}
test.d(30): Deprecation: Symbol core.time.days is not visible
from module test.d because it is privately imported in module
systime
From,
Vino.B
As far as I understand it, days are located in core.time.
import core.time : days;
std.datetime.systime: days Deprecation message
Hi All,
Request your help on the below Deprecation message.
import std.datetime.systime: Clock, days, SysTime;
void main (int AgeSize) {
int AgeSize = 1
auto ct2 = Clock.currTime(), st2 = ct2 + days(-AgeSize);
}
test.d(30): Deprecation: Symbol core.time.days is not visible
from module test.d because it is privately imported in module
systime
From,
Vino.B
Re: HTTP-methods and encoding
On Saturday, 7 April 2018 at 13:02:39 UTC, Vindex wrote: There is an error on some sites when using HTTP-methods (std.net.curl.get, std.net.curl.post): std.encoding.EncodingException@std/encoding.d(2505): Unrecognized Encoding: utf8 Is there a beautiful way around it? For the GET-method I use the download() and readText(). But for the POST-method I can not come up with an alternative solution. That's weird. std.net.curl should be able to handle UTF-8. What content are you trying to download/post? Maybe you can open a bug report for it? In any case, you might want to checkout requests: https://github.com/ikod/dlang-requests It's by far more convenient to use than std.net.curl
Re: How to destruct class instances allocated by a Region-allocator over a single GC block
On Saturday, 7 April 2018 at 07:50:37 UTC, Eduard Staniloiu wrote: On Friday, 6 April 2018 at 21:49:37 UTC, Per Nordlöw wrote: On Tuesday, 3 April 2018 at 09:14:28 UTC, Eduard Staniloiu wrote: So, say `reg` is your allocator, your workflow would be auto obj = reg.make!Type(args); /* do stuff */ reg.dispose(obj); // If Type has a __dtor, it will call obj.__dtor // and then reg.deallocate(obj) If I do sucessive calls to reg.make!X where X are different kinds of classes of different sizes how does reg.dispose(obj) figure out at which address(es) (where emplace filled in the data) the objects reside? It can't figure out. With custom allocators you have to manually do the memory management, so the responsibility of when and which object needs to be destroyed falls on the user of the custom allocator. IMHO, such a complexity should be wrapped in a typed allocation layer. Have Andrei spoken anything about `TypedAllocator`(s) to wrap this complexity?
Re: Function template argument deduction
On Saturday, 7 April 2018 at 14:02:55 UTC, Paul Backus wrote:
Interesting. Looks like this is an issue with aliases, because
I get the error with this code too:
--- test.d
import std.typecons: Tuple, tuple;
alias Pair(T) = Tuple!(T, T);
void foo(T)(Pair!T p)
{
return;
}
unittest {
Pair!int x = tuple(1, 2);
foo(x);
}
---
Looks like this is the same problem described in issue 1807:
https://issues.dlang.org/show_bug.cgi?id=1807
I'm not sure what D was like when Martin Nowak made the most
recent comment on that issue, in 2012, but alias templates do
have their own dedicated language construct now, so maybe this is
worth revisiting.
Re: dustmite watch shell script (.test vs .lookahead.*)
On Friday, 6 April 2018 at 15:42:04 UTC, Vladimir Panteleev wrote: On Friday, 6 April 2018 at 15:35:59 UTC, Anonymouse wrote: The dustmite wiki[0] lists the following example script for use to monitor the reduction progress: Here's a more complete version that also works with -j: https://gist.github.com/CyberShadow/2e8f01895c248111c171e982313bb008 Thanks!
Re: Function template argument deduction
On Saturday, 7 April 2018 at 06:26:24 UTC, Uknown wrote:
What I did notice though is that when
`string list2string(T)(List!T list)` was changed to
`string list2string(T)(VariantN!(16LU, Nil, Tuple!(T, "head",
This*, "tail")) list)`
The compiler correctly deduce `T` to be `int`
Interesting. Looks like this is an issue with aliases, because I
get the error with this code too:
--- test.d
import std.typecons: Tuple, tuple;
alias Pair(T) = Tuple!(T, T);
void foo(T)(Pair!T p)
{
return;
}
unittest {
Pair!int x = tuple(1, 2);
foo(x);
}
---
Store struct tuple of alias and access members through it?
(Please read at the very bottom what I'd like to achieve)
Is it possible to return the member of a struct by its .tupleof
index?
I know that it would work on a struct value, but I'd like it to
work on the type's tupleof:
```
struct S { int i;}
S s;
// below leads to: Error: need this for s1 of type uint
// writeln(/*somehow access s via the S tupleof? */ S.tupleof[0]);
// vs
writeln(s.tupleof[0]);
```
See below example to make the intention a bit clearer:
https://run.dlang.io/gist/6fdb01ddd78b14f8b9a94ac951580cb8
```
struct S
{
uint s1;
ushort s2;
}
interface IParam
{}
template Param(T)
{
static if (isBasicType!T)
alias members = AliasSeq!();
else
alias members = AliasSeq!(T.tupleof);
class Param : IParam
{
T m;
this(T m)
{
this.m = m;
}
IParam opIndex(size_t i)
{
// Something like this possible?
// return this.m.members[i]; // <
how???
// This works but feels needless.
static foreach (j, t; members)
if (i == j)
{
return new
Param!(typeof(members[j]))(__traits(getMember, this.m,
members[j].stringof)); // <-
members[j].stringof feels ugly just to get the member that should
be stored in 'members' already...
}
return null;
}
}
}
```
The reason why I don't want `m.tupleof[i]` is because later I'd
like to consider bitfields within the struct. This means I'd have
to also consider the member functions of the struct and
potentially return them.
E.g.
```
struct S
{
int s1;
int s2() { return 3; }
}
```
and then I'd like to have
alias members = (s1, s2) // pseudo code..
so I could return
S s;
s.members[1]; // would evaluate the function s2 and return the
value
--
In the end I would like to accomplish the following:
Provide access to contained bitfields and members of a struct in
the order they
appear in the struct via an index.
I hope I made a somewhat decent job in explaining what I'm trying
to accomplish.
Please let me know if anything is unclear.
HTTP-methods and encoding
There is an error on some sites when using HTTP-methods (std.net.curl.get, std.net.curl.post): std.encoding.EncodingException@std/encoding.d(2505): Unrecognized Encoding: utf8 Is there a beautiful way around it? For the GET-method I use the download() and readText(). But for the POST-method I can not come up with an alternative solution.
Re: Game and GC
gc causes unpredictabilities in performance*. With games it tends to be worst case performance that matters. I would reccomend using std.experimental.allocator (even if you still use the default GC backed allocator). This will allow you to swap out your allocator for a more specialised one as your requirements become more concrete. auto foo = new CustomStruct(); becomes auto foo = allocator.make!CustomStruct(); The next thing you probably want is @nogc - Last time I checked getting IAllocator objects are a bit tricky to use in @nogc code. Currently I am using https://github.com/radcapricorn/alloctraits to get around this limitation (You will still need an allocator that doesn't use the GC, I use Mallocator for test purposes). * The GC itself is deterministic, but it is really easy to write code that triggers GC pauses at times that is difficult track down. On Sat, Apr 7, 2018 at 7:55 AM, Chris Katko via Digitalmars-d-learn < digitalmars-d-learn@puremagic.com> wrote: > I'm in the same boat. I do games. But I love D's syntax and template > power. So I'm doing a full experiment. > > Honestly, if D is that big a liability, you'll encounter it long before > it's "too late" to port it to C++. > > Last night I had stuttering issues, but I realized there was a single, > C-function, being called too many times (and never deallocating!). > > But previously, I've also had stutter issues. Now granted, I test on a > "crap" laptop 2 GB RAM / Celeron processor. But it'll be 60 FPS ... then > spike down. If this happens again with my current project, what I'm going > to do, is hack the open source garbage collector to fire off an > event/console message EVERY TIME it actually pauses to collect. Because > it's possible the GC isn't actually the problem, or, some simple change to > a line of code may prevent the GC from being a problem. > > That said, there's also @nogc (but that's also a bit of a lie because they > never tell you that ANY THREAD running GC code can pause ALL THREADS for a > collection.) > > But if you're making games, you should really be using static pools > anyway. What's the MAXIMUM number of objects/trees/maps your game will have > at a time? It's simple (regardless of D, C, Python, or Lua). Static. Pools. > Basically, you just allocate at startup a simple fixed-length array for all > your objects. That way, you're never asking the OS for memory = Never > needing the garbage collector. If you don't use all that memory? Who cares. > RAM is cheap. And if your program CAN swell in size, that means your > low-end PCs will fail without knowing why. > > So you just put all your objects in fixed length arrays of size > MAX_OBJECTS, MAX_ENEMIES, MAX_ITEMS, etc. And deleting an object is as > simple as erasing it, or marking it as "bool is_deleted = true;" and adding > a new object is simply finding the first "is_deleted" and re-running the > constructor / re-using the carcass of the dead object. > > 99% of AAA studios use static pools. Now technically, static pools are > "chunks" of fixed length arrays. So you could have one pool for a "map", > and start loading another pool for the next map you're going to enter, and > then when you finally transfer to the next map, you then free the static > pool by marking it as deleted. And repeat as necessary. So it's a very > macro-level amount of allocations. We're talking like, less than a dozen > actual entities. (Depends on gametype, of course. But the > order-of-magnitude helps convey it.) >
Re: How to destruct class instances allocated by a Region-allocator over a single GC block
On Friday, 6 April 2018 at 21:49:37 UTC, Per Nordlöw wrote: On Tuesday, 3 April 2018 at 09:14:28 UTC, Eduard Staniloiu wrote: So, say `reg` is your allocator, your workflow would be auto obj = reg.make!Type(args); /* do stuff */ reg.dispose(obj); // If Type has a __dtor, it will call obj.__dtor // and then reg.deallocate(obj) If I do sucessive calls to reg.make!X where X are different kinds of classes of different sizes how does reg.dispose(obj) figure out at which address(es) (where emplace filled in the data) the objects reside? It can't figure out. With custom allocators you have to manually do the memory management, so the responsibility of when and which object needs to be destroyed falls on the user of the custom allocator.
