Re: Asynchronous Programming and Eventhandling in D
On Tuesday, 5 July 2016 at 20:38:53 UTC, Eugene Wissner wrote:
On Tuesday, 5 July 2016 at 08:24:43 UTC, O/N/S wrote:
Hi ("Grüss Gott")
I like the asynchronous events in Javascript.
Is something similar possible in D?
Found Dragos Carp's asynchronous library
(https://github.com/dcarp/asynchronous).
Are there any more integrated (in Phobos/in D) ways to work
asynchronously?
An example: One server ask a second server to calculate
something big.
The first server continues with his work, until the answer
come back from the second.
And so on...
Using threads or fibers would be a way, but has not the same
elegancy like the Javascript way. (To avoid discussions: D is
better ;-)
Greetings from Munich,
Ozan
Can you describe what would you like to see more concretly. I
know js but how is it supposed to work for D? Maybe you can
give some example, kind of pseudo code? It would help me much
to build a concept and maybe we will see someday something
usable in this area :)
Hi,
I took a typical example (using jquery lib, other js-libs have
similar calls)
$.ajax({
type: "POST",
url: myurl,
dataType: "JSON",
data: $myData,
async: true,
success: function(success) { ... },
error: function(error) { ... }
});
If response is successful -> call function(success)
If response has errors -> call function(error)
In the meanwhile the code can continue without waiting for the
response.
In D-style it would look like
...do something..
funcAjax(["type":"POST"...], delegateSuccess, delegateError);
- or -
funcAjax(["type":"POST"...], (success) => {...}, (error) =>
{...});
...continue without waiting...
Why delegates not functions? Because responses could require
changes in data, which are outside (yeah, I know, it's also
technically possible with function, but that's not the idea
behind)
Greetings from Munich, Ozan
Re: Asynchronous Programming and Eventhandling in D
On Wednesday, 6 July 2016 at 14:57:08 UTC, chmike wrote:
On Wednesday, 6 July 2016 at 11:33:53 UTC, Eugene Wissner wrote:
The only reason libev was choosen is that it is the simplest
implementation I know about. A few C files. I had an
educational purpose: I wanted to see how an event loop works
on low level. Asyncio was for me no-go, since I've not written
a lot in C++, and can only read the code a bit. So I'm not
hanging on libev. The only another implementation would be the
python event library. It is also pure C code and it was much
more cleaner written than libev on the first sight.
Your project and work is valuable on many aspects. I didn't
mean to depreciate it.
Now there are two problems with my work:
1) The first is something we all are tired to talk about:
manual memory management. I make a proof of concept and am
writing the code that is 100% marked as @nogc. It has side
effects. For example I allocate exceptions and thay should be
freed after catching - it is something, phobos doesn't do. As
said it is an experiment; I would like to see how it works.
2) Performance. The main reason I started the writing at all
is that the existing implementations seem to have only
performance as criterium. Performance is super important, but
not on the cost of design, usability and extensibility. For
example in vibe.d (and libasync) everything possible is
defined as structs, everything that would be interesting to
extend is final; and after it you go to phobos and see a
"workaround". For example Mallocator is a struct but you have
to be sure, it is an allocator. How do you force the right
interface?
That is a valid point. I know it is hard to get the best
performance and optimal API design at the same time. The reason
the methods are final is to avoid the overhead of virtual
method indirection.
static if (hasMember!(Allocator, "deallocate"))
{
return impl.deallocate(b);
}
else
{
return false;
}
Somethinkg like this would be ok for C. But for a language
with interfaces, it is ugly design independent of how it
performs. Everything I say here is IMHO.
This would mean we need a new design pattern that supports both.
Except these two points I'm interested in some kind of
collective work aswell. It is very difficult as one man job. I
didn't know other people are also working on similar
implementations. Nice to know.
Are you aware of any benchmark tools in other languages that
could be used?
The benchmark tools available are mainly testing web servers.
And the exact operation pattern is not very clear. One of such
benchmark tool is wrk [https://github.com/wg/wrk] that measure
HTTP request speed. The problem is that it measure the
performance of the I/O and HTTP handling.
Here are some benchmark results:
* https://www.techempower.com/benchmarks/
* https://github.com/nanoant/WebFrameworkBenchmark
How can D be worse than Java ?
My strategy would be to split the problem. First get the async
I/O optimal. Then get HTTP handling optimal, and finally get
database interaction optimal (the optimal async I/O should
help). An investigation on the methods used by Java/undertow to
get these performances could help.
I would suggest to implement a benchmark client doing some
predefined patterns of I/O operations similar to web
interactions or the most common types of interactions. And a
server implemented in C using native system I/O operations.
This server implementation would then be our reference.
What would be measured is how much slower our different D
implementations are relative to the C reference implementation.
This will allow us to have a benchmarking test that doesn't
depend that much of the hardware.
Yes. it would be a way to go.
As I see Kore performs pretty well. One could write bindings
first and then begin to rewrite step for step. On every
development step you have a working http server and you can use
these http-benchmark tools for the benchmarking.
I have to look in it in the next weeks
Re: Asynchronous Programming and Eventhandling in D
On Wednesday, 6 July 2016 at 11:33:53 UTC, Eugene Wissner wrote:
The only reason libev was choosen is that it is the simplest
implementation I know about. A few C files. I had an
educational purpose: I wanted to see how an event loop works on
low level. Asyncio was for me no-go, since I've not written a
lot in C++, and can only read the code a bit. So I'm not
hanging on libev. The only another implementation would be the
python event library. It is also pure C code and it was much
more cleaner written than libev on the first sight.
Your project and work is valuable on many aspects. I didn't mean
to depreciate it.
Now there are two problems with my work:
1) The first is something we all are tired to talk about:
manual memory management. I make a proof of concept and am
writing the code that is 100% marked as @nogc. It has side
effects. For example I allocate exceptions and thay should be
freed after catching - it is something, phobos doesn't do. As
said it is an experiment; I would like to see how it works.
2) Performance. The main reason I started the writing at all is
that the existing implementations seem to have only performance
as criterium. Performance is super important, but not on the
cost of design, usability and extensibility. For example in
vibe.d (and libasync) everything possible is defined as
structs, everything that would be interesting to extend is
final; and after it you go to phobos and see a "workaround".
For example Mallocator is a struct but you have to be sure, it
is an allocator. How do you force the right interface?
That is a valid point. I know it is hard to get the best
performance and optimal API design at the same time. The reason
the methods are final is to avoid the overhead of virtual method
indirection.
static if (hasMember!(Allocator, "deallocate"))
{
return impl.deallocate(b);
}
else
{
return false;
}
Somethinkg like this would be ok for C. But for a language with
interfaces, it is ugly design independent of how it performs.
Everything I say here is IMHO.
This would mean we need a new design pattern that supports both.
Except these two points I'm interested in some kind of
collective work aswell. It is very difficult as one man job. I
didn't know other people are also working on similar
implementations. Nice to know.
Are you aware of any benchmark tools in other languages that
could be used?
The benchmark tools available are mainly testing web servers. And
the exact operation pattern is not very clear. One of such
benchmark tool is wrk [https://github.com/wg/wrk] that measure
HTTP request speed. The problem is that it measure the
performance of the I/O and HTTP handling.
Here are some benchmark results:
* https://www.techempower.com/benchmarks/
* https://github.com/nanoant/WebFrameworkBenchmark
How can D be worse than Java ?
My strategy would be to split the problem. First get the async
I/O optimal. Then get HTTP handling optimal, and finally get
database interaction optimal (the optimal async I/O should help).
An investigation on the methods used by Java/undertow to get
these performances could help.
I would suggest to implement a benchmark client doing some
predefined patterns of I/O operations similar to web interactions
or the most common types of interactions. And a server
implemented in C using native system I/O operations. This server
implementation would then be our reference.
What would be measured is how much slower our different D
implementations are relative to the C reference implementation.
This will allow us to have a benchmarking test that doesn't
depend that much of the hardware.
Re: Asynchronous Programming and Eventhandling in D
On Wednesday, 6 July 2016 at 08:39:03 UTC, chmike wrote:
On Tuesday, 5 July 2016 at 20:38:53 UTC, Eugene Wissner wrote:
On Tuesday, 5 July 2016 at 08:24:43 UTC, O/N/S wrote:
Hi ("Grüss Gott")
I like the asynchronous events in Javascript.
Is something similar possible in D?
Found Dragos Carp's asynchronous library
(https://github.com/dcarp/asynchronous).
Are there any more integrated (in Phobos/in D) ways to work
asynchronously?
An example: One server ask a second server to calculate
something big.
The first server continues with his work, until the answer
come back from the second.
And so on...
Using threads or fibers would be a way, but has not the same
elegancy like the Javascript way. (To avoid discussions: D is
better ;-)
Greetings from Munich,
Ozan
Servus,
I'm currently rewriting the base skeleton of libev in D (only
for linux for now) for web development aswell. And the next
step would be data structures, basic server, futures and yo
on...
I was working with dcarp's asynchronous and i found it very
very good. It is till now the best I've seen in D for async
programming ( I mean its design and usability).
Can you describe what would you like to see more concretly. I
know js but how is it supposed to work for D? Maybe you can
give some example, kind of pseudo code? It would help me much
to build a concept and maybe we will see someday something
usable in this area :)
The problem I see with libev is that it isn't compatible with
the CPIO API of windows.
The C++ boost asio library, on the contrary, is compatible with
the select/epoll/kqueue model and the Windows CPIO model.
This is the reason I started on a D implementation of asio I
called dasio [https://github.com/chmike/dasio]. Unfortunately
my alimentary work didn't leave me any time to make progress on
it. The only thing I manage to implement so far is asio's error
code system.
I'm glad to see other people see an interest to work on this.
We definitely should find a way to combine our efforts. That is
already a significant work made with the other libraries.
My feeling is that providing support for very efficient IO in
Phobos might have a strong impact on D's visibility and
adoption for backend applications (e.g. servers). Performance
is a very strong argument for adoption is such context.
A list of requirements has been already published on the wiki.
What I think is now missing is a benchmarking tool so that we
can get numbers for each async lib implementation that we can
also compare with a raw C implementation using the native
functions.
The only reason libev was choosen is that it is the simplest
implementation I know about. A few C files. I had an educational
purpose: I wanted to see how an event loop works on low level.
Asyncio was for me no-go, since I've not written a lot in C++,
and can only read the code a bit. So I'm not hanging on libev.
The only another implementation would be the python event
library. It is also pure C code and it was much more cleaner
written than libev on the first sight.
Now there are two problems with my work:
1) The first is something we all are tired to talk about: manual
memory management. I make a proof of concept and am writing the
code that is 100% marked as @nogc. It has side effects. For
example I allocate exceptions and thay should be freed after
catching - it is something, phobos doesn't do. As said it is an
experiment; I would like to see how it works.
2) Performance. The main reason I started the writing at all is
that the existing implementations seem to have only performance
as criterium. Performance is super important, but not on the cost
of design, usability and extensibility. For example in vibe.d
(and libasync) everything possible is defined as structs,
everything that would be interesting to extend is final; and
after it you go to phobos and see a "workaround". For example
Mallocator is a struct but you have to be sure, it is an
allocator. How do you force the right interface?
static if (hasMember!(Allocator, "deallocate"))
{
return impl.deallocate(b);
}
else
{
return false;
}
Somethinkg like this would be ok for C. But for a language with
interfaces, it is ugly design independent of how it performs.
Everything I say here is IMHO.
Except these two points I'm interested in some kind of collective
work aswell. It is very difficult as one man job. I didn't know
other people are also working on similar implementations. Nice to
know.
Are you aware of any benchmark tools in other languages that
could be used?
Re: Asynchronous Programming and Eventhandling in D
On 06/07/2016 8:39 PM, chmike wrote:
On Tuesday, 5 July 2016 at 20:38:53 UTC, Eugene Wissner wrote:
On Tuesday, 5 July 2016 at 08:24:43 UTC, O/N/S wrote:
Hi ("Grüss Gott")
I like the asynchronous events in Javascript.
Is something similar possible in D?
Found Dragos Carp's asynchronous library
(https://github.com/dcarp/asynchronous).
Are there any more integrated (in Phobos/in D) ways to work
asynchronously?
An example: One server ask a second server to calculate something big.
The first server continues with his work, until the answer come back
from the second.
And so on...
Using threads or fibers would be a way, but has not the same elegancy
like the Javascript way. (To avoid discussions: D is better ;-)
Greetings from Munich,
Ozan
Servus,
I'm currently rewriting the base skeleton of libev in D (only for
linux for now) for web development aswell. And the next step would be
data structures, basic server, futures and yo on...
I was working with dcarp's asynchronous and i found it very very good.
It is till now the best I've seen in D for async programming ( I mean
its design and usability).
Can you describe what would you like to see more concretly. I know js
but how is it supposed to work for D? Maybe you can give some example,
kind of pseudo code? It would help me much to build a concept and
maybe we will see someday something usable in this area :)
The problem I see with libev is that it isn't compatible with the CPIO
API of windows.
The C++ boost asio library, on the contrary, is compatible with the
select/epoll/kqueue model and the Windows CPIO model.
This is the reason I started on a D implementation of asio I called
dasio [https://github.com/chmike/dasio]. Unfortunately my alimentary
work didn't leave me any time to make progress on it. The only thing I
manage to implement so far is asio's error code system.
I'm glad to see other people see an interest to work on this. We
definitely should find a way to combine our efforts. That is already a
significant work made with the other libraries.
My feeling is that providing support for very efficient IO in Phobos
might have a strong impact on D's visibility and adoption for backend
applications (e.g. servers). Performance is a very strong argument for
adoption is such context.
A list of requirements has been already published on the wiki.
What I think is now missing is a benchmarking tool so that we can get
numbers for each async lib implementation that we can also compare with
a raw C implementation using the native functions.
Hmm interesting way to do errors, might have to steal that for
http://spew.cf
Re: Asynchronous Programming and Eventhandling in D
On Tuesday, 5 July 2016 at 20:38:53 UTC, Eugene Wissner wrote:
On Tuesday, 5 July 2016 at 08:24:43 UTC, O/N/S wrote:
Hi ("Grüss Gott")
I like the asynchronous events in Javascript.
Is something similar possible in D?
Found Dragos Carp's asynchronous library
(https://github.com/dcarp/asynchronous).
Are there any more integrated (in Phobos/in D) ways to work
asynchronously?
An example: One server ask a second server to calculate
something big.
The first server continues with his work, until the answer
come back from the second.
And so on...
Using threads or fibers would be a way, but has not the same
elegancy like the Javascript way. (To avoid discussions: D is
better ;-)
Greetings from Munich,
Ozan
Servus,
I'm currently rewriting the base skeleton of libev in D (only
for linux for now) for web development aswell. And the next
step would be data structures, basic server, futures and yo
on...
I was working with dcarp's asynchronous and i found it very
very good. It is till now the best I've seen in D for async
programming ( I mean its design and usability).
Can you describe what would you like to see more concretly. I
know js but how is it supposed to work for D? Maybe you can
give some example, kind of pseudo code? It would help me much
to build a concept and maybe we will see someday something
usable in this area :)
The problem I see with libev is that it isn't compatible with the
CPIO API of windows.
The C++ boost asio library, on the contrary, is compatible with
the select/epoll/kqueue model and the Windows CPIO model.
This is the reason I started on a D implementation of asio I
called dasio [https://github.com/chmike/dasio]. Unfortunately my
alimentary work didn't leave me any time to make progress on it.
The only thing I manage to implement so far is asio's error code
system.
I'm glad to see other people see an interest to work on this. We
definitely should find a way to combine our efforts. That is
already a significant work made with the other libraries.
My feeling is that providing support for very efficient IO in
Phobos might have a strong impact on D's visibility and adoption
for backend applications (e.g. servers). Performance is a very
strong argument for adoption is such context.
A list of requirements has been already published on the wiki.
What I think is now missing is a benchmarking tool so that we can
get numbers for each async lib implementation that we can also
compare with a raw C implementation using the native functions.
Re: Asynchronous Programming and Eventhandling in D
On Tuesday, 5 July 2016 at 08:24:43 UTC, O/N/S wrote:
Hi ("Grüss Gott")
I like the asynchronous events in Javascript.
Is something similar possible in D?
Found Dragos Carp's asynchronous library
(https://github.com/dcarp/asynchronous).
Are there any more integrated (in Phobos/in D) ways to work
asynchronously?
An example: One server ask a second server to calculate
something big.
The first server continues with his work, until the answer come
back from the second.
And so on...
Using threads or fibers would be a way, but has not the same
elegancy like the Javascript way. (To avoid discussions: D is
better ;-)
Greetings from Munich,
Ozan
Servus,
I'm currently rewriting the base skeleton of libev in D (only for
linux for now) for web development aswell. And the next step
would be data structures, basic server, futures and yo on...
I was working with dcarp's asynchronous and i found it very very
good. It is till now the best I've seen in D for async
programming ( I mean its design and usability).
Can you describe what would you like to see more concretly. I
know js but how is it supposed to work for D? Maybe you can give
some example, kind of pseudo code? It would help me much to build
a concept and maybe we will see someday something usable in this
area :)
