How to send ownerTid into a parallel foreach loop?

2020-06-26 Thread adnan338 via Digitalmars-d-learn
I have a list of files to download, and I want to download them 
in parallel. At the end of each of those parallel download I want 
to send the main thread a message from the parallel loop.


import std.concurrency, std.parallelism;

string[] files = ["a", "b", "c"];

void download(string[] links)
{
auto owner = ownerTid();
foreach (link; links.parallel())
{
// something
owner.send(false);
}
owner.send(true);
}

void main()
{
// I do not want my main thread to freeze
spawn(, files);
bool done = false;
while (!done)
receive((bool isDone) { done = isDone; });
}

But the compiler says:
Error: static assert:  "Aliases to mutable thread-local data not 
allowed."
source/app.d(19,7):instantiated from here: spawn!(void 
function(string[]), string[])


How should I go around this? I think the parallel block is being 
restricted from sending messages to the owner.

Here,


Re: Calling C functions

2020-06-26 Thread Jacob Carlborg via Digitalmars-d-learn

On 2020-06-26 18:54, Denis wrote:


OK, now this makes sense.

I tested calling the same callback function directly from D: it compiled 
and worked correctly. So at least prefixing the callback function with 
`extern(C)` doesn't prevent the rest of the D program from calling it too.


No, of course not. How would you otherwise call your `cfunc` function 
from your original example ;)


--
/Jacob Carlborg


Re: Calling C functions

2020-06-26 Thread Denis via Digitalmars-d-learn

On Friday, 26 June 2020 at 08:15:27 UTC, Jacob Carlborg wrote:

On Friday, 26 June 2020 at 00:30:22 UTC, Denis wrote:



  extern(C) void cfunc(void function(int));
  extern(C) void dcallback(int x) {...} <-- Why extern(C)?
  cfunc();

Can this be rewritten, dropping the prefix from the second 
line? If not, it would be helpful to know why "extern(C)" is 
needed here too.


No, it cannot be dropped. `extern(C)` is required because C and 
D are using different calling conventions (D functions are also 
mangled). For example, D (at least DMD and LDC) are passing the 
arguments to the function in reverse.


OK, now this makes sense.

I tested calling the same callback function directly from D: it 
compiled and worked correctly. So at least prefixing the callback 
function with `extern(C)` doesn't prevent the rest of the D 
program from calling it too.


(2) Is there a way to restrict the invocation of a linked C 
function to one specific D function?


[...]

For functions nested in a D language construct (class, struct, 
function) the compiler will always use the D mangling, instead 
of the C mangling. In theory it would be possible to workaround 
that by forcing the mangled name using `pragma(mangle)`, but 
for some reason the compiler doesn't allow `pragma(mangle)` 
inside a function body, on a nested function declaration.


You can wrap up everything in a struct, as follows:


I see.

Thank you very much for these explanations and code -- the 
insights are very helpful.


Denis


Re: Recursive delegate inside template?

2020-06-26 Thread drathier via Digitalmars-d-learn

On Friday, 26 June 2020 at 15:10:07 UTC, Adam D. Ruppe wrote:

On Friday, 26 June 2020 at 14:12:00 UTC, drathier wrote:

I'm trying to get this to compile, without much luck:


You might be able to make it a function:


bool foo()
{
auto fn(A)()
{
A delegate(A) fn;
fn = delegate A(A a)
{
return fn(a);
};
return fn;
}

return fn!(bool)()(true);
}



I tried this, but it gives me `Error: non-constant expression 
` which I don't understand. Is ctfe kicking in even though 
this is inside a function?


```
void foo2()
{
auto fn(A) = std.functional.toDelegate(({
A fn(A)(A a)
{
return a;
}

return !(A);
}()));
}
```

but I need the function to be polymorphic, so I need the 
template.


I don't really understand your need though, can you post more 
context?


I'm generating code, compiling a strict pure functional language 
to D. The code that I'm trying to compile is this example 
function:

```
fn a =
let
_ = fn a
in a
```
which gets the type `forall a. a -> a` inferred.

That's something like
```
A delegate(A) fn(A) = delegate A (A a)
{
auto _ = fn!(A)(a);
return a;
};
```
in D syntax, if it would compile. Well, it compiles, but the 
linker fails `error: TLS relocation against invalid instruction`.


It's a simplified example that still tests these features:
- recursion
- polymorphism
- functions as values (closures/delegates)



Re: Recursive delegate inside template?

2020-06-26 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jun 26, 2020 at 02:12:00PM +, drathier via Digitalmars-d-learn 
wrote:
> I'm trying to get this to compile, without much luck:
> 
> ```
> bool foo()
> {
> template fn(A)
> {
> A delegate(A) fn;
> fn = delegate A(A a)
> {
> return fn(a);
> };
> }
> 
> return fn!(bool)(true);
> }
> ```
> 
> What am I missing here? How can I define a recursive delegate inside
> another function like this?
[...]

Sounds like you're looking for the equivalent of the Y combinator,
perhaps?

https://en.wikipedia.org/wiki/Fixed-point_combinator#Y_combinator


T

-- 
Never trust an operating system you don't have source for! -- Martin Schulze


Re: Unused template arguments; what type to use?

2020-06-26 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jun 26, 2020 at 01:40:57PM +, Simen Kjærås via Digitalmars-d-learn 
wrote:
> On Friday, 26 June 2020 at 13:21:25 UTC, drathier wrote:
> > How can I tell the compiler that I will never create a value of type
> > X, while still being able to write code that uses it? Using void as
> > a template parameter is where I started, but I still need to be able
> > to declare variables inside this unreachable function, like `T foo;`
> > even when `T == void`. Can I get any closer to what I want than an
> > empty struct?
> 
> Depends on what you care about, I guess. A final abstract class has
> been my go-to on a few occasions. I'd argue the empty struct is better
> in most cases, and a forward-declared struct with no implementation
> might work in some cases.
[...]

You can also use void[0], which has size 0, unlike an empty struct which
has size 1.


T

-- 
The best compiler is between your ears. -- Michael Abrash


Re: Downloading files over TLS

2020-06-26 Thread Jacob Carlborg via Digitalmars-d-learn

On 2020-06-26 15:16, Adam D. Ruppe wrote:

Yeah, I've been wanting to change that and use the native apis for years 
but like I just haven't been able to figure out the documentation of them.


That would be nice.

That's the problem of everything Apple makes. Kinda drives me nuts 
trying to keep up with their endless churn and constantly Think 
Different campaigns.


Hehe, yeah.

https://docs.microsoft.com/en-us/windows/win32/winhttp/winhttp-sessions-overview 



I guess that is more middle level but it isn't too hard to use. I think 
there's a flat out url -> bytes function too but I don't remember what 
it is.


Regardless, the Windows functions may look familiar if you have done 
AJAX - that was based on an IE object which was based on the Windows API.


Thanks, I'll take a look.

--
/Jacob Carlborg


Re: Recursive delegate inside template?

2020-06-26 Thread Adam D. Ruppe via Digitalmars-d-learn

On Friday, 26 June 2020 at 14:12:00 UTC, drathier wrote:

I'm trying to get this to compile, without much luck:


You might be able to make it a function:


bool foo()
{
auto fn(A)()
{
A delegate(A) fn;
fn = delegate A(A a)
{
return fn(a);
};
return fn;
}

return fn!(bool)()(true);
}

but I need the function to be polymorphic, so I need the 
template.


I don't really understand your need though, can you post more 
context?


Re: Downloading files over TLS

2020-06-26 Thread Jacob Carlborg via Digitalmars-d-learn

On 2020-06-26 14:43, User wrote:

It is possible to statically link libcurl into your application. No need 
to use OpenSSL as libcurl can be built with SChannel.


That sounds good.

It looks like I remembered wrong. std.net.curl uses `dlopen` on libcurl, 
not OpenSSL. This might be less of an issue assuming libcurl is built 
with the platform provided TLS implementation. Just make sure it's 
possible to statically link libcurl, without using `dlopen`.


--
/Jacob Carlborg


Re: Downloading files over TLS

2020-06-26 Thread Jacob Carlborg via Digitalmars-d-learn

On 2020-06-26 14:41, Kagamin wrote:


Maybe just start wget or something like that?


The point was to avoid runtime dependencies.

Since you want the latest certificate storage, you intend to support 
only the latest system. Many root certificates will timeout now.


I didn't say the latest certificate storage. I said the platform 
provided. If the system provided certificate expires I guess a lot of 
things won't work. An alternative would be to both ship with a 
certificate and be able to use the system provided one and try both



https://docs.microsoft.com/en-us/windows/win32/winhttp/about-winhttp


I'll take a look, thanks.

--
/Jacob Carlborg


Re: isInputRange not satisfied even if all individual conditions are satisfied

2020-06-26 Thread Paul Backus via Digitalmars-d-learn

On Friday, 26 June 2020 at 13:53:46 UTC, Johannes Loher wrote:

Am 26.06.20 um 15:35 schrieb ag0aep6g:
`isInputRange!R` fails because it has no knowledge of your 
free `empty` function. Without `empty`, `R` is obviously not a 
range.


Ah, OK, that makes sense. It's kind of sad though because it 
really limits the extensibility of existing types with UFCS. Do 
you know if there is a way around this?


You can use my recently-released Dub package `addle` to make your 
UFCS extensions visible to std.range.


Dub: https://code.dlang.org/packages/addle
Announcement: 
https://forum.dlang.org/thread/yruztugitygumwcsm...@forum.dlang.org


Recursive delegate inside template?

2020-06-26 Thread drathier via Digitalmars-d-learn

I'm trying to get this to compile, without much luck:

```
bool foo()
{
template fn(A)
{
A delegate(A) fn;
fn = delegate A(A a)
{
return fn(a);
};
}

return fn!(bool)(true);
}
```

What am I missing here? How can I define a recursive delegate 
inside another function like this? I even got a gold linker error 
from run.dlang.io at one point: "TLS relocation against invalid 
instruction".


Removing the template works fine:

```
bool foo()
{
alias A = bool;
A delegate(A) fn;
fn = delegate A(A a) { return fn(a); };

return fn(true);
}
```
but I need the function to be polymorphic, so I need the template.


Re: isInputRange not satisfied even if all individual conditions are satisfied

2020-06-26 Thread ag0aep6g via Digitalmars-d-learn

On 26.06.20 15:35, ag0aep6g wrote:
`isInputRange!R` fails because it has no knowledge of your free `empty` 
function. Without `empty`, `R` is obviously not a range.


To be clear: It's the same with `front` and `popFront`. You can't 
implement any range primitives as free functions.


It only works for arrays because those implementations are part of 
std.range.


Re: isInputRange not satisfied even if all individual conditions are satisfied

2020-06-26 Thread Johannes Loher via Digitalmars-d-learn
Am 26.06.20 um 15:35 schrieb ag0aep6g:
> `isInputRange!R` fails because it has no knowledge of your free `empty`
> function. Without `empty`, `R` is obviously not a range.

Ah, OK, that makes sense. It's kind of sad though because it really
limits the extensibility of existing types with UFCS. Do you know if
there is a way around this?


Re: Unused template arguments; what type to use?

2020-06-26 Thread Simen Kjærås via Digitalmars-d-learn

On Friday, 26 June 2020 at 13:21:25 UTC, drathier wrote:
How can I tell the compiler that I will never create a value of 
type X, while still being able to write code that uses it? 
Using void as a template parameter is where I started, but I 
still need to be able to declare variables inside this 
unreachable function, like `T foo;` even when `T == void`. Can 
I get any closer to what I want than an empty struct?


Depends on what you care about, I guess. A final abstract class 
has been my go-to on a few occasions. I'd argue the empty struct 
is better in most cases, and a forward-declared struct with no 
implementation might work in some cases.


--
  Simen


Re: isInputRange not satisfied even if all individual conditions are satisfied

2020-06-26 Thread ag0aep6g via Digitalmars-d-learn

On 26.06.20 15:09, Johannes Loher wrote:

import std.meta : allSatisfy, staticMap;
import std.traits : allSameType;
import std.range : isInputRange, ElementType, empty;

[...]

@property bool empty(TypeArgs...)(auto ref scope SumType!(TypeArgs) r)
 if (allSatisfy!(isInputRange, TypeArgs) && TypeArgs.length > 0)
{
 return r.match!(staticMap!(EmptyLambda, TypeArgs));
}

[...]

enum bool myIsInputRange(R) =
 is(typeof(R.init) == R)
 && is(ReturnType!((R r) => r.empty) == bool)
 && is(typeof((return ref R r) => r.front))
 && !is(ReturnType!((R r) => r.front) == void)
 && is(typeof((R r) => r.popFront));

void main() {

[...]

 import std.traits : ReturnType;

[...]

 alias R = SumType!(typeof(i), typeof(o));

 // all individual conditions of `isInputRange` are satisfied
 static assert(is(typeof(R.init) == R));
 static assert(is(ReturnType!((R r) => r.empty) == bool));

[...]

 // but `isInputRange` is not satisfied
 static assert(!isInputRange!(R));

 // and neither is a local copy
 static assert(!myIsInputRange!(R));
}


`isInputRange!R` fails because it has no knowledge of your free `empty` 
function. Without `empty`, `R` is obviously not a range.


`myIsInputRange!R` fails because you forgot to import `ReturnType` in 
module scope. You're importing it locally in `main`, so the check passes 
there.


Unused template arguments; what type to use?

2020-06-26 Thread drathier via Digitalmars-d-learn
How can I tell the compiler that I will never create a value of 
type X, while still being able to write code that uses it? Using 
void as a template parameter is where I started, but I still need 
to be able to declare variables inside this unreachable function, 
like `T foo;` even when `T == void`. Can I get any closer to what 
I want than an empty struct?


Re: Downloading files over TLS

2020-06-26 Thread Adam D. Ruppe via Digitalmars-d-learn

On Friday, 26 June 2020 at 10:12:09 UTC, Jacob Carlborg wrote:

* Arsd [4]. Relies on OpenSSL


Yeah, I've been wanting to change that and use the native apis 
for years but like I just haven't been able to figure out the 
documentation of them.


Though for plain download, on Windows there's a high level 
function you can just call that's relatively easy... just the 
rest of the module needs low level access so it isn't super 
useful.


And of course Mac does it differently, I don't know much about 
there.



The main problem is that it has been deprecated.


That's the problem of everything Apple makes. Kinda drives me 
nuts trying to keep up with their endless churn and constantly 
Think Different campaigns.


* Are there any high level APIs, like NSURLSession, on Windows 
that can be used to download files?


https://docs.microsoft.com/en-us/windows/win32/winhttp/winhttp-sessions-overview

I guess that is more middle level but it isn't too hard to use. I 
think there's a flat out url -> bytes function too but I don't 
remember what it is.


Regardless, the Windows functions may look familiar if you have 
done AJAX - that was based on an IE object which was based on the 
Windows API.


isInputRange not satisfied even if all individual conditions are satisfied

2020-06-26 Thread Johannes Loher via Digitalmars-d-learn
Recently, I read about problems regarding the API design of ranges (not
being able to make them const, differences between class and struct
based ranges etc.).

One of the issues that came up what the fact that sometimes it is useful
to use the class wrappers `InputRangeObject` etc. if you need to decide
at runtime which range you want to use. I ran into this into a personal
project in the past and this also was how I solved the issue.

Another suggestion in that thread was to simply use choose, which I
guess should work, but I had trouble getting it to work in my particular
case.

However, a third option came to my mind: Why not simply use a sumtype to
return either one range or the other? In order to make it more
convenient to use afterwards, it would be useful if we could regard a
sumtype of ranges as a range itself. That should be no problem because
we just need to delegate all function calls to the range api to the
actual range that sits inside the sumtype.

So I tried to implement this and it kind of seems to work (I can call
the individual functions on the sumtype using UFCS) but for some reason,
`isInputRange` evaluates to `false`. What makes it even weirder is the
fact that all individual conditions that are part of `isInputRange` seem
to be fulfilled. Could somebody explain to me what is going on? Here is
the code:

https://run.dlang.io/gist/93bcc196f73b0a7c7aa1851beef59c22 (currently
give me a 502 though)

```
#!/usr/bin/env dub
/+ dub.sdl:
name "sumtype-range"
dependency "sumtype" version="~>0.9.4"
+/

module sumtype_range;

import std.meta : allSatisfy, staticMap;
import std.traits : allSameType;
import std.range : isInputRange, ElementType, empty;

import sumtype;

private template EmptyLambda(T) if (isInputRange!T)
{
alias EmptyLambda = (ref T t) => t.empty;
}

@property bool empty(TypeArgs...)(auto ref scope SumType!(TypeArgs) r)
if (allSatisfy!(isInputRange, TypeArgs) && TypeArgs.length > 0)
{
return r.match!(staticMap!(EmptyLambda, TypeArgs));
}

private template FrontLambda(T) if (isInputRange!T)
{
alias FrontLambda = (ref T t) => t.front;
}

@property ElementType!(TypeArgs[0]) front(TypeArgs...)(auto ref scope
SumType!(TypeArgs) r)
if (allSatisfy!(isInputRange, TypeArgs)
&& allSameType!(staticMap!(ElementType, TypeArgs)) &&
TypeArgs.length > 0)
{
return r.match!(staticMap!(FrontLambda, TypeArgs));
}

private template PopFrontLambda(T) if (isInputRange!T)
{
alias PopFrontLambda = (ref T t) => t.popFront();
}

void popFront(TypeArgs...)(auto ref scope SumType!(TypeArgs) r)
if (allSatisfy!(isInputRange, TypeArgs) && TypeArgs.length > 0)
{
return r.match!(staticMap!(PopFrontLambda, TypeArgs));
}

enum bool myIsInputRange(R) =
is(typeof(R.init) == R)
&& is(ReturnType!((R r) => r.empty) == bool)
&& is(typeof((return ref R r) => r.front))
&& !is(ReturnType!((R r) => r.front) == void)
&& is(typeof((R r) => r.popFront));

void main() {
import std.range : iota, only;
import std.stdio : writeln;
import std.traits : ReturnType;

auto i = iota(4);
auto o = only(1);
alias R = SumType!(typeof(i), typeof(o));

// all individual conditions of `isInputRange` are satisfied
static assert(is(typeof(R.init) == R));
static assert(is(ReturnType!((R r) => r.empty) == bool));
static assert(is(typeof((return ref R r) => r.front)));
static assert(!is(ReturnType!((R r) => r.front) == void));
static assert(is(typeof((R r) => r.popFront)));

// but `isInputRange` is not satisfied
static assert(!isInputRange!(R));

// and neither is a local copy
static assert(!myIsInputRange!(R));
}
```


Re: Downloading files over TLS

2020-06-26 Thread Kagamin via Digitalmars-d-learn

On Friday, 26 June 2020 at 10:12:09 UTC, Jacob Carlborg wrote:
Downloading files over TLS. This seems that it's something that 
should be quite simple to do. My high level goals are 
cross-platform and easy distribution. I don't need anything 
fancy just a simple API like this:


download("https://url.com;, "/local/file");


Maybe just start wget or something like that?

* Network [6]. This is an Apple specific library provided by 
the platform. This is the recommend alternative to 
SecureTransport. The problem is that this is not just an 
alternative TLS implementation, it's a completely different 
alternative to BSD sockets. The API is completely different and 
will require some extra layers to to provide a cross-platform 
API. This means that I cannot use any of the existing library 
to just add TLS, it will be a completely different 
implementation, which might be ok. Another big problem is that 
it only available on macOS 10.14 and later. I have not decided 
yet if this is acceptable or not


Since you want the latest certificate storage, you intend to 
support only the latest system. Many root certificates will 
timeout now.


* Are there any high level APIs, like NSURLSession, on Windows 
that can be used to download files?


https://docs.microsoft.com/en-us/windows/win32/winhttp/about-winhttp


Re: Downloading files over TLS

2020-06-26 Thread User via Digitalmars-d-learn

On Friday, 26 June 2020 at 10:12:09 UTC, Jacob Carlborg wrote:


Windows:
I don't know that much of this platform.

* std.net.curl and basically all other options already 
mentioned relies on OpenSSL, which is not provided by the 
platform


* SChannel. As far as I know, this the the platform provided 
implementation of TLS on Windows.


* Are there any high level APIs, like NSURLSession, on Windows 
that can be used to download files?




It is possible to statically link libcurl into your application. 
No need to use OpenSSL as libcurl can be built with SChannel.




Re: Downloading files over TLS

2020-06-26 Thread ikod via Digitalmars-d-learn

On Friday, 26 June 2020 at 11:41:27 UTC, Jacob Carlborg wrote:

On Friday, 26 June 2020 at 11:10:27 UTC, ikod wrote:


[...]


Oh, it's that bad. That's disappointing.


[...]


I'm using a script (written in D) in my tool, DStep [1][2], to 
identify which versions of libclang is available and to select 
between dynamic link (not using `dlopen`) and static linking of 
libclang. It works pretty well.






[1] 
https://github.com/jacob-carlborg/dstep/blob/master/configure.d
[2] 
https://github.com/jacob-carlborg/dstep/blob/master/dub.json#L34-L41


Thanks, will try to improve this (at least to add static openssl 
linking, as this is not first time somebody asked about it)




--
/Jacob Carlborg




Re: Downloading files over TLS

2020-06-26 Thread Jacob Carlborg via Digitalmars-d-learn

On Friday, 26 June 2020 at 11:10:27 UTC, ikod wrote:


Hello,

re `requests` - it uses dlopen (and variants for OSX and 
Windows, see 
https://github.com/ikod/dlang-requests/blob/master/source/requests/ssl_adapter.d#L50). The reason for dlopen is simple - compatibility with both openssl ver 1.0 and 1.1 (which have incompatible interfaces). To solve this problem I expose common interface for Requests internal needs, and detect and use different underlying openssl interfaces depending on library version.


Oh, it's that bad. That's disappointing.

I'm sure it is possible to detect library version at build 
time, and then use static linking. It just require some time 
investment to find most acceptable solution on how to do this.


I'm using a script (written in D) in my tool, DStep [1][2], to 
identify which versions of libclang is available and to select 
between dynamic link (not using `dlopen`) and static linking of 
libclang. It works pretty well.


Re Windows - I'd be happy to use native SecureChannel but I 
have zero windows programming experience. I'm completely open 
for any help in this field.


I don't have much experience with programming on Windows either. 
macOS is my main platform.


Bottom line - problem with SSL/TLS libraries lies in 
incompatibility between platforms and even inside the single 
library.


Yes. I'm trying to identify the best solution, which ideally 
requires the least amount of work.


[1] 
https://github.com/jacob-carlborg/dstep/blob/master/configure.d
[2] 
https://github.com/jacob-carlborg/dstep/blob/master/dub.json#L34-L41


--
/Jacob Carlborg


Re: Downloading files over TLS

2020-06-26 Thread ikod via Digitalmars-d-learn

On Friday, 26 June 2020 at 10:12:09 UTC, Jacob Carlborg wrote:
Downloading files over TLS. This seems that it's something that 
should be quite simple to do. My high level goals are 
cross-platform and easy distribution. I don't need anything 
fancy just a simple API like this:


download("https://url.com;, "/local/file");




Is there anything obvious I'm missing or why does this seem so 
difficult? Do I have too high expectations and requirements?


[1] https://code.dlang.org/packages/requests
[2] https://code.dlang.org/packages/vibe-d
[3] https://code.dlang.org/packages/hunt
[4] https://code.dlang.org/packages/arsd-official%3Ahttp
[5] 
https://developer.apple.com/documentation/security/secure_transport

[6] https://developer.apple.com/documentation/network
[7] 
https://developer.apple.com/documentation/foundation/nsurlsession

[8] https://code.dlang.org/packages/botan

--
/Jacob Carlborg


Hello,

re `requests` - it uses dlopen (and variants for OSX and Windows, 
see 
https://github.com/ikod/dlang-requests/blob/master/source/requests/ssl_adapter.d#L50). The reason for dlopen is simple - compatibility with both openssl ver 1.0 and 1.1 (which have incompatible interfaces). To solve this problem I expose common interface for Requests internal needs, and detect and use different underlying openssl interfaces depending on library version.


I'm sure it is possible to detect library version at build time, 
and then use static linking. It just require some time investment 
to find most acceptable solution on how to do this.


Re Windows - I'd be happy to use native SecureChannel but I have 
zero windows programming experience. I'm completely open for any 
help in this field.


Bottom line - problem with SSL/TLS libraries lies in 
incompatibility between platforms and even inside the single 
library.




Downloading files over TLS

2020-06-26 Thread Jacob Carlborg via Digitalmars-d-learn
Downloading files over TLS. This seems that it's something that 
should be quite simple to do. My high level goals are 
cross-platform and easy distribution. I don't need anything fancy 
just a simple API like this:


download("https://url.com;, "/local/file");

Because of these goals, I have a few requirements, which I don't 
think are unreasonable:


* Cross-platform (macOS, Linux, Windows, possible FreeBSD)

* Minimal runtime requirements (I don't mind a few compile time 
requirements). I also don't mind runtime requirements that are 
guaranteed to exist


* Ideally using the platform provided TLS implementation, or at 
least platform provided certificates (I've been bitten several 
times due to old certificates making request fail)


* On Linux I would really like to have a fully statically linked 
binary, this will make distribution easier


* Due to the previous requirement, I think using musl will be 
required. For most parts glibc works fine for static linking, but 
I think in the case of networking it does not work. The DNS 
lookup (or something like that) doesn't work with static linking


I think the main problem of all this is picking the right TLS 
implementation.


The obvious candidate is std.net.curl, but I'm going to explain 
why that is not ideal, at least not how it's currently 
implemented. std.net.curl relies on OpenSSL, which is deprecated 
on several platforms. On several platforms it's not the main, 
platform provided TLS implementation. As far as I know, it also 
provides its own certificates and doesn't rely on the ones 
provided by the platform.


I going to break down some alternatives per platform.

macOS:
* std.net.curl. Depends on libcurl and OpenSSL. OpenSSL has been 
deprecated on macOS for many years. In one wants to use OpenSSL 
anyway, one need to ship it with the binary (statically link it). 
For other problems, see above


* requests [1]. This is a potential better alternative than 
std.net.curl since it doesn't rely on libcurl. But, it still 
relies on OpenSSL, so it has the same problems as std.net.curl


* vibe.d [2]. Does not depend on libcurl, but does depend on 
OpenSSL. But, it also supports Botan, which is promising. I'll 
discuss Botan separately


* Hunt [3]. Same problem as the other ones, relies on OpenSSL

* Arsd [4]. Relies on OpenSSL

* SecureTransport [5]. This is an Apple specific library provided 
by the platform. It seems to more or less follow the same idea as 
OpenSSL, it's independent on the transport layer and works with 
BSD sockets. Initially this looks perfect, it's provided by the 
platform and uses certificates provided by the platform. The main 
problem is that it has been deprecated. It also only supports TLS 
up to version 1.2 and since it's deprecated, it's likely that it 
won't support any newer versions


* Network [6]. This is an Apple specific library provided by the 
platform. This is the recommend alternative to SecureTransport. 
The problem is that this is not just an alternative TLS 
implementation, it's a completely different alternative to BSD 
sockets. The API is completely different and will require some 
extra layers to to provide a cross-platform API. This means that 
I cannot use any of the existing library to just add TLS, it will 
be a completely different implementation, which might be ok. 
Another big problem is that it only available on macOS 10.14 and 
later. I have not decided yet if this is acceptable or not


* NSURLSession [7]. This is an  Apple specific Objective-C class, 
provided by the platform. It's higher level than the Network 
framework and is available on all versions of macOS. This will 
again require a some extra layers to hide the platform specific 
API. There's also a problem with LDC which has very limited 
support for Objective-C interoperability.


Linux:
As far as I know, OpenSSL (or some version of it, like LibreSSL) 
is what's provided by the platform. So I guess that has to be 
acceptable on Linux.


* std.net.curl. The main problem with std.net.curl on Linux is 
that it uses `dlopen` to load the TLS library. `dlopen` doesn't 
work for a statically linked executable. Side note, `dlopen` 
doesn't work on iOS (for other reasons) so it would be nice if 
this could be fixed


* requests. Seems to do that same as std.net.curl, uses `dlopen`

* For the other alternatives (mentioned in the macOS section), I 
haven't investigated if they use `dlopen` or not


Windows:
I don't know that much of this platform.

* std.net.curl and basically all other options already mentioned 
relies on OpenSSL, which is not provided by the platform


* SChannel. As far as I know, this the the platform provided 
implementation of TLS on Windows.


* Are there any high level APIs, like NSURLSession, on Windows 
that can be used to download files?


Botan:

This seems like a good candidate. It even exists a full D port of 
it [8].


* This is not used by any specific platform but it can use the 
platform provided 

Re: Arduino and MCU Support

2020-06-26 Thread Dukc via Digitalmars-d-learn

On Thursday, 25 June 2020 at 03:00:04 UTC, Dylan Graham wrote:
I'm currently making an automatic transmission controller with 
Arduino. C++ just has too many traps that I keep falling into. 
Since stability is critical (if the code screws up at 100km/h 
I'm dead), I'd rather use a sane language like D.


No, don't! Regardless of the language, you should never trust 
your safety on one single program. See 
https://www.digitalmars.com/articles/b39.html


Realistically, doing such a controller cannot be one man/woman 
endeavour.


Re: Calling C functions

2020-06-26 Thread Jacob Carlborg via Digitalmars-d-learn

On Friday, 26 June 2020 at 00:30:22 UTC, Denis wrote:

I have a two questions about calling C functions from D.

(1) When passing a D callback to a C function, is there a way 
to write the code without having to prefix the callback 
declaration with "extern(C)"?


It's not a big deal adding the prefix to the D function 
declaration. It just seems odd to me to prefix D code with 
"extern(C)". For example, the following code works:


  extern(C) void cfunc(void function(int));
  extern(C) void dcallback(int x) {...} <-- Why extern(C)?
  cfunc();

Can this be rewritten, dropping the prefix from the second 
line? If not, it would be helpful to know why "extern(C)" is 
needed here too.


No, it cannot be dropped. `extern(C)` is required because C and D 
are using different calling conventions (D functions are also 
mangled). For example, D (at least DMD and LDC) are passing the 
arguments to the function in reverse.


(2) Is there a way to restrict the invocation of a linked C 
function to one specific D function?


If the C header is defined in one of the core.stdc libraries, 
the import statement can either be global or inside a specific 
D function -- both work. In contrast, when the C function 
prototype is written directly into the D program (as above), 
the linker complains unless this declaration is made global. If 
it's possible to restrict the scope of the C function to just 
one D function, I'll take advantage.


For functions nested in a D language construct (class, struct, 
function) the compiler will always use the D mangling, instead of 
the C mangling. In theory it would be possible to workaround that 
by forcing the mangled name using `pragma(mangle)`, but for some 
reason the compiler doesn't allow `pragma(mangle)` inside a 
function body, on a nested function declaration.


You can wrap up everything in a struct, as follows:

struct printf
{
pragma(mangle, "printf") extern (C) private static int 
printf(in char*, ...);


static int opCall(Args...)(Args args)
{
return printf(args);
}
}

void main()
{
printf("asd\n".ptr);
}

The `printf` function can be called from anywhere within the 
module, but not outside the module.


--
/Jacob Carlborg