Re: Introducing vibe.d!
Am 26.04.2012 23:59, schrieb bearophile:
Sönke Ludwig:
See http://vibed.org/ for more information and some example
I see the code:
import vibe.d;
...
static this()
{
listenTcp(7, (conn){ conn.write(conn) });
}
Isn't it better to use this?
import vibe.all;
And in the last line is a semicolon missing?
listenTcp(7, (conn){ conn.write(conn); });
Bye,
bearophile
Ah yes, the semicolon got lost while transforming conn =
conn.write(conn) to (conn){ conn.write(conn); } because the lamdba
syntax was a bit surprising there.
The import vibe.d; is actually just a play on the frameworks name and
just should be easy to remember. In addition to importing all modules,
it also imports a module containing the main function, so this is a
special beast. But there is actually a import vibe.vibe; which is the
equivalent to the vibe.all.
Sönke
Re: Introducing vibe.d!
On Friday, 27 April 2012 at 02:19:23 UTC, Andrei Alexandrescu wrote: On 4/26/12 3:30 PM, Jesse Phillips wrote: On Thursday, 26 April 2012 at 22:05:29 UTC, Robert Clipsham wrote: On 26/04/2012 21:46, Sönke Ludwig wrote: vibe.d This looks awesome! Also on reddit: http://www.reddit.com/r/programming/comments/su9la/vibed_asynchronous_io_that_doesnt_get_in_your_way/ and hacker news: https://news.ycombinator.com/item?id=3896197 Now look what you did, you killed it. Are the Vibe people on this? Though the intent is very nice, this is not looking great right now. Andrei It's back up. It went down while he was asleep. Regards, Brad Anderson
Re: Introducing vibe.d!
Am 27.04.2012 04:19, schrieb Andrei Alexandrescu: On 4/26/12 3:30 PM, Jesse Phillips wrote: On Thursday, 26 April 2012 at 22:05:29 UTC, Robert Clipsham wrote: On 26/04/2012 21:46, Sönke Ludwig wrote: vibe.d This looks awesome! Also on reddit: http://www.reddit.com/r/programming/comments/su9la/vibed_asynchronous_io_that_doesnt_get_in_your_way/ and hacker news: https://news.ycombinator.com/item?id=3896197 Now look what you did, you killed it. Are the Vibe people on this? Though the intent is very nice, this is not looking great right now. Andrei It's back up now and I've made a small watchdog script to avoid this in the future. The problem is that I'm currently using a vibe based reverse proxy instead of nginx and it rarely has an uncaught exception that unfortunately has never been logged with stack trace. I wish I could have made this part rock solid before going live but someone else annouced it before me so it was too late.
Re: Introducing vibe.d!
On 2012-04-26 22:46, Sönke Ludwig wrote: During the last few months, we have been working on a new framework for general I/O and especially for building extremely fast web apps. It combines asynchronous I/O with core.thread's great fibers to build a convenient, blocking API which can handle insane amounts of connections due to the low memory and computational overhead. Some of its key fatures are: Wow, that's impressive. -- /Jacob Carlborg
Re: Introducing vibe.d!
Great work! This is the the type of server side frameworks we need. -- Paulo On Thursday, 26 April 2012 at 20:46:41 UTC, Sönke Ludwig wrote: During the last few months, we have been working on a new framework for general I/O and especially for building extremely fast web apps. It combines asynchronous I/O with core.thread's great fibers to build a convenient, blocking API which can handle insane amounts of connections due to the low memory and computational overhead. Some of its key fatures are: - Very fast but no endless callback chains as in node.js and similar frameworks - Concise API that tries to be as efficient and intuitive as possible - Built-in HTTP server and client with support for HTTPS, chunked and compressed transfers, keep-alive connections, Apache-style logging, a reverse-proxy, url routing and more - Jade based HTML/XML template system with compile-time code generation for the fastest dynamic page generation times possible - Built-in support for MongoDB and Redis databases - WebSocket support - Natural Json and Bson handling - A package manager for seemless use of extension libraries See http://vibed.org/ for more information and some example applications (there are some things in the works such as an etherpad clone and an NNTP server). vibe.d is in a working state and enters its first beta-phase now to stabilize the current feature set. After that, a small list of additional features is planned before the 1.0 release. The framework can be downloaded or GIT cloned from http://vibed.org/ and is distributed under the terms of the MIT license. Note that the website including the blog is fully written in vibe and provides the first stress test for the implementation. Regards, Sönke
Re: Introducing vibe.d!
On Thursday, 26 April 2012 at 20:46:41 UTC, Sönke Ludwig wrote: During the last few months, we have been working on a new framework for general I/O and especially for building extremely fast web apps. It combines asynchronous I/O with core.thread's great fibers to build a convenient, blocking API which can handle insane amounts of connections due to the low memory and computational overhead. Some of its key fatures are: - Very fast but no endless callback chains as in node.js and similar frameworks - Concise API that tries to be as efficient and intuitive as possible - Built-in HTTP server and client with support for HTTPS, chunked and compressed transfers, keep-alive connections, Apache-style logging, a reverse-proxy, url routing and more - Jade based HTML/XML template system with compile-time code generation for the fastest dynamic page generation times possible - Built-in support for MongoDB and Redis databases - WebSocket support - Natural Json and Bson handling - A package manager for seemless use of extension libraries See http://vibed.org/ for more information and some example applications (there are some things in the works such as an etherpad clone and an NNTP server). vibe.d is in a working state and enters its first beta-phase now to stabilize the current feature set. After that, a small list of additional features is planned before the 1.0 release. The framework can be downloaded or GIT cloned from http://vibed.org/ and is distributed under the terms of the MIT license. Note that the website including the blog is fully written in vibe and provides the first stress test for the implementation. Regards, Sönke I had to copy the included .lib files into bin in order to build the examples but so far, so good. This is awesome. Regards, Brad Anderson
Re: Introducing vibe.d!
I had to copy the included .lib files into bin in order to build the examples but so far, so good. This is awesome. Regards, Brad Anderson There is some really strange behavior of Windows batch files where it sometimes fails with environment variables set with quotes and sometimes without... Now I changed the line SET LIBDIR=..\lib\win-i386 in run_example.cmd from quoted to non-quoted and for me it seems to work. Any idea what might cause this erratic behavior?
Re: Introducing vibe.d!
On Friday, 27 April 2012 at 07:00:23 UTC, Sönke Ludwig wrote: I had to copy the included .lib files into bin in order to build the examples but so far, so good. This is awesome. Regards, Brad Anderson There is some really strange behavior of Windows batch files where it sometimes fails with environment variables set with quotes and sometimes without... Now I changed the line SET LIBDIR=..\lib\win-i386 in run_example.cmd from quoted to non-quoted and for me it seems to work. Any idea what might cause this erratic behavior? I haven't a clue. Are these diet templates static? I was looking for more information but this page doesn't exist: http://vibed.org/templates
Re: Introducing vibe.d!
Am 27.04.2012 09:08, schrieb Brad Anderson: On Friday, 27 April 2012 at 07:00:23 UTC, Sönke Ludwig wrote: I had to copy the included .lib files into bin in order to build the examples but so far, so good. This is awesome. Regards, Brad Anderson There is some really strange behavior of Windows batch files where it sometimes fails with environment variables set with quotes and sometimes without... Now I changed the line SET LIBDIR=..\lib\win-i386 in run_example.cmd from quoted to non-quoted and for me it seems to work. Any idea what might cause this erratic behavior? I haven't a clue. Are these diet templates static? I was looking for more information but this page doesn't exist: http://vibed.org/templates Yes, they are compiled to D code at compile time but you can use dynamic expressions and statements inside. The result is a bunch of calls of the form: 'write(htmlheadtitle); write(htmlEscape(myDynamicVariable)); The features page* has a paragraph about them but the syntax reference is still missing. I will add a notice that the Jade reference** can be used instead for now. * http://vibed.org/features#diet-templates ** https://github.com/visionmedia/jade
Re: Introducing vibe.d!
Sounds great! And the site's very speedy :) I'm especially excited about this: (Work-in-progress) An integrated load balancer is able to dynamically compile, run and test new processes before switching them live after the code has been modified. A few questions: - Does the web framework work with other web servers, or is it tied to the built-in server? - I don't suppose you have a feature comparison between the built-in server and Nginx? Is it one o' them fancy C10k servers? (Not that I'm likely to ever get that kind of traffic outside of a DDoS attack ;) ) - Why static this? Seems like a strange choice since it'll run before the main that (I assume) vibed automatically provides - and in an undefined order relative to all other module ctors.
Re: Introducing vibe.d!
Sönke Ludwig wrote: During the last few months, we have been working on a new framework for general I/O and especially for building extremely fast web apps. It combines asynchronous I/O with core.thread's great fibers to build a convenient, blocking API which can handle insane amounts of connections due to the low memory and computational overhead. Some of its key fatures are: - Very fast but no endless callback chains as in node.js and similar frameworks - Concise API that tries to be as efficient and intuitive as possible - Built-in HTTP server and client with support for HTTPS, chunked and compressed transfers, keep-alive connections, Apache-style logging, a reverse-proxy, url routing and more - Jade based HTML/XML template system with compile-time code generation for the fastest dynamic page generation times possible - Built-in support for MongoDB and Redis databases - WebSocket support - Natural Json and Bson handling - A package manager for seemless use of extension libraries See http://vibed.org/ for more information and some example applications (there are some things in the works such as an etherpad clone and an NNTP server). vibe.d is in a working state and enters its first beta-phase now to stabilize the current feature set. After that, a small list of additional features is planned before the 1.0 release. The framework can be downloaded or GIT cloned from http://vibed.org/ and is distributed under the terms of the MIT license. Note that the website including the blog is fully written in vibe and provides the first stress test for the implementation. Regards, Sönke Sönke, vibed is truly amazing! I am interested in the web server's internal architecture. I always wanted to do an implementation of a web server using a form of asymmetric, multi-process event-driven architecture. A web server which utilises fibers. It would be nice if you explain the web-server architecture of vibed in more details. Keep up with good work! Regards
Re: Introducing vibe.d!
Am 27.04.2012 10:01, schrieb Nick Sabalausky: Sounds great! And the site's very speedy :) I'm especially excited about this: (Work-in-progress) An integrated load balancer is able to dynamically compile, run and test new processes before switching them live after the code has been modified. Me too, me too, it would have saved my ass last night when the server went down ;) A few questions: - Does the web framework work with other web servers, or is it tied to the built-in server? You can put it behind an nginx reverse proxy (beginning with the new v1.2 that supports HTTP/1.1). You could also just use everything except the HTTP server component and use classic CGI/FastCGI to communicate to the Web server. However, there is currently no FastCGI server built in, so something like libfcgi would have to do. - I don't suppose you have a feature comparison between the built-in server and Nginx? Is it one o' them fancy C10k servers? (Not that I'm likely to ever get that kind of traffic outside of a DDoS attack ;) ) We still have a more comprehensive benchmark on the table but it seemed to get along happily with about 60MB of RAM usage during a C10k test. The average request time went down to about 6s if I remember correctly. The test was using a dual-core Atom server over a WiFI connection so the results may have been skewed a litte. I'm also planning to implement a core driver based on libuv in addition to the current libevent based one, which should gain a little more performance. In terms of HTTP/1.1 implementation it is most importantly lacking multipart form uploads (will be added shortly). Otherwise the protocol supports most HTTP features. I'll put a more detailed comparison on my TODO list. - Why static this? Seems like a strange choice since it'll run before the main that (I assume) vibed automatically provides - and in an undefined order relative to all other module ctors. The order of module constructors should be defined to be a valid order wrt. the import tree right? The main reason is to have a lean and mean way to write small applications. A default main() is compiled in which handles command line parsing, connection to a load-balancer/reverse proxy etc. However, it is also possible to just implement a custom main() and start from there - you just have to include vibe.vibe instead of vibe.d so that the built-in main() is not compiled. I will put an example on the website later.
Re: Introducing vibe.d!
On 04/27/2012 10:01 AM, Nick Sabalausky wrote: ... - Why static this? Seems like a strange choice since it'll run before the main that (I assume) vibed automatically provides - and in an undefined order relative to all other module ctors. This is not true. The order is defined as far it matters for static initialisation. I think this is perfectly fine.
Re: Introducing vibe.d!
Am 27.04.2012 10:10, schrieb Dmitry Olshansky:
On 27.04.2012 0:46, Sönke Ludwig wrote:
During the last few months, we have been working on a new
framework for general I/O and especially for building
extremely fast web apps. It combines asynchronous I/O with
core.thread's great fibers to build a convenient, blocking
API which can handle insane amounts of connections due to
the low memory and computational overhead.
Some of its key fatures are:
In short: meet the new boss of the web development :)
Few things I think worth looking at.
1. Add cool auto-magic API generator in Adam's style:
route.wrapModule!(module_name)(/url);
The effect is that it binds /url/function_name for each function found
in module, automatically gets parameters from post (or get) using
arguments names of function. Last but not least it issues necessary
calls to std.conv.to as needed.
Ah and the result is again converted to string (if needed) and packed
into response body.
Example:
module factorial;
long fact(int k)
{
return k 1 ? k*fact(k-1) : 1;
}
Wrapped via wrapModule!(factorial)(/); will work as
/fact?k=10 will out the result in plain-text. MIME type and conversion
should be configurable of course.
Now on the TODO list!
2. By the very nature of your framework you should be well prepared to
small-scale DDoS attacks. But there is much more in DoS assortment these
days. It would be shame if your framework doesn't handle e.g.
slow-writer attack. Imagine attacker spoon-feeds HTTP request to your
server by 1 byte per 10msec. Bump simultaneous request count to few
thousands and your nice event loop freezes to a crawl. You gotta check
upload rates ;)
There is already a HttpServerOptions.maxRequestTime that would help -
but this is currently still ignored. Also, keep-alive connections are
kept open too long right now. But both is on the list.
Maybe something like a LRU limiter that closes old connections in case
of too many open connections would also be a good alternative barrier...
My plan is also to switch all modules to @safe as soon as Phobos and
Druntime have a bit better support. I guess this is type of application
can be considered the prototype for which SafeD is useful.
Re: Introducing vibe.d!
Am 27.04.2012 10:16, schrieb David Nadlinger: On Thursday, 26 April 2012 at 22:25:33 UTC, Vladimir Panteleev wrote: On Thursday, 26 April 2012 at 20:46:41 UTC, Sönke Ludwig wrote: During the last few months, we have been working on a new framework for general I/O and especially for building extremely fast web apps. Very nice, amazing job! This is like my personal plans and hopes coming to life. +1, although I didn't have the chance to look at it in detail yet. How should we submit »bug reports« for the website? David Initially I wanted to have a page-comment feature ready as on dlang.org. But for now I guess the github issue tracker should do the job: https://github.com/rejectedsoftware/vibe.d/issues Sönke
Re: Introducing vibe.d!
Sönke Ludwig slud...@outerproduct.org wrote in message news:jndl9l$26eh$1...@digitalmars.com... We still have a more comprehensive benchmark on the table but it seemed to get along happily with about 60MB of RAM usage during a C10k test. The average request time went down to about 6s if I remember correctly. The test was using a dual-core Atom server over a WiFI connection so the results may have been skewed a litte. I guess I don't have much of a frame of reference, but that certainly sounds very impressive. Especially the RAM usage (you could probably kill lighttpd, from what I've heard about it) and also the fact that was all over WiFi. Have you looked at long-term memory usage? I wonder if D's imprecise GC could be a liability and would need to be ? In terms of HTTP/1.1 implementation it is most importantly lacking multipart form uploads (will be added shortly). Otherwise the protocol supports most HTTP features. I'll put a more detailed comparison on my TODO list. Thanks, I'm anxious to take a look. I'm particulary curious how much of your intended scope does/doesn't overlap with Nginx's domain and it's (rather Apache-competetive) suite of modules: http://wiki.nginx.org/Modules Do you see this as eventually (or even currently) competing with Nginx in general? Or is it more focused on web framework and general async I/O? (Actually that's another thing, how general is the async I/O in it? Is it intended to be used for just...whatever general I/O uses...or is it more intended as a means-to-an-end for web?) The way it's designed, I suppose the features of many of Nginx's modules could simply be implemented using vibed, for example directory listings. I hope it doesn't sound like I'm trying to tear vibed apart. I'm genuinely impressed with it so far. Just trying to get a good feel for it and how best to use it effectively. - Why static this? Seems like a strange choice since it'll run before the main that (I assume) vibed automatically provides - and in an undefined order relative to all other module ctors. The order of module constructors should be defined to be a valid order wrt. the import tree right? Oops, yea. Not sure what I was thinking. However, it is also possible to just implement a custom main() and start from there - you just have to include vibe.vibe instead of vibe.d so that the built-in main() is not compiled. I will put an example on the website later. Cool.
Re: Introducing vibe.d!
Am 27.04.2012 11:06, schrieb Nick Sabalausky: Sönke Ludwigslud...@outerproduct.org wrote in message news:jndl9l$26eh$1...@digitalmars.com... We still have a more comprehensive benchmark on the table but it seemed to get along happily with about 60MB of RAM usage during a C10k test. The average request time went down to about 6s if I remember correctly. The test was using a dual-core Atom server over a WiFI connection so the results may have been skewed a litte. I guess I don't have much of a frame of reference, but that certainly sounds very impressive. Especially the RAM usage (you could probably kill lighttpd, from what I've heard about it) and also the fact that was all over WiFi. Have you looked at long-term memory usage? I wonder if D's imprecise GC could be a liability and would need to be ? I'm starting to monitor it now. Directly after startup, the website is at 32 MB. The traffic has dropped a bit but hopefully it's enough to see something if there is a hidden leak. After the C10k test, the memory usage dropped back to normal if I remember right so there should be nothing too bad in the core. In terms of HTTP/1.1 implementation it is most importantly lacking multipart form uploads (will be added shortly). Otherwise the protocol supports most HTTP features. I'll put a more detailed comparison on my TODO list. Thanks, I'm anxious to take a look. I'm particulary curious how much of your intended scope does/doesn't overlap with Nginx's domain and it's (rather Apache-competetive) suite of modules: http://wiki.nginx.org/Modules From going just over the module index, the features that are in or planned are: - most of Core is there - Basic auth - check - Auto index - part of a separate library that is still in dev. - FastCGI client is planned, possible also a server - Gzip responses (the static file server componen does this) - Headers - check - Limit Requests/Zone is planned - Limit Conn is planned - Log - check - Proxy - check - scgi - maybe instead of FastCGI - Split Clients - easy to do programmatically - Upstream - planned - SSL - check - WebDAV - planned - simplistic SMTP - check Do you see this as eventually (or even currently) competing with Nginx in general? Or is it more focused on web framework and general async I/O? (Actually that's another thing, how general is the async I/O in it? Is it intended to be used for just...whatever general I/O uses...or is it more intended as a means-to-an-end for web?) The way it's designed, I suppose the features of many of Nginx's modules could simply be implemented using vibed, for example directory listings. In general it's more a general application framework or an extension library. But a stand-alone webserver would be an easy project - nothing that I have directly planned myself though. The I/O is supposed to be as general as possible. Right now the focus has been on web applications but we also plan a desktop application that is going to use it for all of its I/O. For this I'm also looking into how to integrate the event loop with the OS message queue efficiently on all platforms. There are still some things missing for the libevent driver but with the upcoming libuv driver everything that libuv provides should be available.
Re: Introducing vibe.d!
Am 27.04.2012 10:26, schrieb Andrej Mitrovic: On 4/27/12, Sönke Ludwigslud...@outerproduct.org wrote: Now I changed the line SET LIBDIR=..\lib\win-i386 in run_example.cmd from quoted to non-quoted and for me it seems to work. Maybe try: SET LIBDIR=..\lib\win-i386 IOW quotes after SET. Thanks, that also seems to work. I will try that next as soon as the current non-quoted version fails for someone.
Re: Introducing vibe.d!
I am not sure if you're aware of Flask, Flask is a microframework for Python. It provides something called Blueprints, you can register e.g. routes to this Blueprint and to use them you've to add them to the main application. This makes code way more readable! Also some kind of after_request and before_request handlers would be great, you could use them e.g. fetching a user from the database and providing it to the handler (over a thread-local?).
Re: Introducing vibe.d!
Can't wait to go home and get my greedy hands on it. Sounds.. amazing. Hell, if all of this stuff really works and works on par with nginx+fastcgi performance, it can be the very killer app for D waited so long.
Re: Introducing vibe.d!
Excellent work Sönke! Vibe.d seems very promising :) I've played with vibe.d with a hello project as described in the document, and downloaded vibenotes and vibeblog from git and poked around a little. vibe.d gives me a very smooth experience, and I'll try learn about it more :) two little things to note: 1. In the tutorial it didn't say how to start vibe.d project. I spent some minutes to find it: by running vibe at the root of a project. Maybe its clear to state that in the tutorial 2. It seems that diet templates doesn't support soft tab. I looked into the code, and in indentLevel(line) it only dectects '\t'. Will vibe.d support soft tab?
Re: Introducing vibe.d!
Am 27.04.2012 11:57, schrieb David:
I am not sure if you're aware of Flask, Flask is a microframework for
Python.
It provides something called Blueprints, you can register e.g.
routes to this Blueprint and to use them you've to add them to the
main application. This makes code way more readable!
Also some kind of after_request and before_request handlers would be
great, you could use them e.g. fetching a user from the database and
providing it to the handler (over a thread-local?).
I didn't know flask before and the website seems to be down currently ;)
But is the UrlRouter (see http://vibed.org/docs#http-routing and
http://vibed.org/api/vibe.http.router#UrlRouter) something equivalent?
It allows you to specify multiple patterns that are matched in sequence
until a request handler writes a response:
auto r = new UrlRouter;
r.any(*, performAuth);
r.get(/, renderHomePage);
listenHttp(settings, r);
This would first always match performAuth, which could check the user
credentials and write out a message if the user is not authenticated.
Otherwise it just does nothing and the router will continue to the next
match, which is renderHomePage() in this case for GET /.
Whats not directly possible in the router is something like
after_request. But you can do it easily by wrapping the call:
auto r = new UrlRouter;
//...
listenHttp(settings, (req, res){
before_request(req, res);
r.handleRequest(req, res);
after_request(req, res);
});
Re: Introducing vibe.d!
Am 27.04.2012 13:07, schrieb Puming: Excellent work Sönke! Vibe.d seems very promising :) Thanks :) I've played with vibe.d with a hello project as described in the document, and downloaded vibenotes and vibeblog from git and poked around a little. vibe.d gives me a very smooth experience, and I'll try learn about it more :) two little things to note: 1. In the tutorial it didn't say how to start vibe.d project. I spent some minutes to find it: by running vibe at the root of a project. Maybe its clear to state that in the tutorial It definitely needs some improvements. My idea would be to change it to a step-by-step tutorial and also add support for something like vibe init appname to generate the basic app skeleton. 2. It seems that diet templates doesn't support soft tab. I looked into the code, and in indentLevel(line) it only dectects '\t'. Will vibe.d support soft tab? I put an item in the TODO list to detect the indent style from the first indented line. This indent style would then be used for the rest of the document.
Re: Introducing vibe.d!
Am 27.04.2012 13:18, schrieb Sönke Ludwig:
Am 27.04.2012 11:57, schrieb David:
I am not sure if you're aware of Flask, Flask is a microframework for
Python.
It provides something called Blueprints, you can register e.g.
routes to this Blueprint and to use them you've to add them to the
main application. This makes code way more readable!
Also some kind of after_request and before_request handlers would be
great, you could use them e.g. fetching a user from the database and
providing it to the handler (over a thread-local?).
I didn't know flask before and the website seems to be down currently ;)
Yes :(
Admin paused the server, too much IO or something.
But is the UrlRouter (see http://vibed.org/docs#http-routing and
http://vibed.org/api/vibe.http.router#UrlRouter) something equivalent?
It allows you to specify multiple patterns that are matched in sequence
until a request handler writes a response:
auto r = new UrlRouter;
r.any(*, performAuth);
r.get(/, renderHomePage);
listenHttp(settings, r);
This would first always match performAuth, which could check the user
credentials and write out a message if the user is not authenticated.
Otherwise it just does nothing and the router will continue to the next
match, which is renderHomePage() in this case for GET /.
This is useful!
Whats not directly possible in the router is something like
after_request. But you can do it easily by wrapping the call:
auto r = new UrlRouter;
//...
listenHttp(settings, (req, res){
before_request(req, res);
r.handleRequest(req, res);
after_request(req, res);
});
Awesome!
Re: Introducing vibe.d!
On Friday, 27 April 2012 at 08:47:23 UTC, Sönke Ludwig wrote: Initially I wanted to have a page-comment feature ready as on dlang.org. But for now I guess the github issue tracker should do the job: https://github.com/rejectedsoftware/vibe.d/issues Oh, and another thing, is vpm live yet? I though about starting to hack on a Socket.io implementation for Vibe this weekend… David
Re: Introducing vibe.d!
Am 27.04.2012 13:57, schrieb David Nadlinger: On Friday, 27 April 2012 at 08:47:23 UTC, Sönke Ludwig wrote: Initially I wanted to have a page-comment feature ready as on dlang.org. But for now I guess the github issue tracker should do the job: https://github.com/rejectedsoftware/vibe.d/issues Oh, and another thing, is vpm live yet? I though about starting to hack on a Socket.io implementation for Vibe this weekend… David It's live but currently projects can only be registered manually and there is no user web frontend - not a big deal though. But it still needs some testing and misses some features (e.g. package signing, updating and machine-wide installs). Btw. there is already a WebSocket implementation in vibe.http.websockets, although still undocumented (vibenotes uses it). Jan, who has written it also planned on doing a fallback for older browsers using long polling. But socket.io compatibility was not on the table yet..
Re: Introducing vibe.d!
Sönke, vibed is truly amazing! I am interested in the web server's internal
architecture. I always wanted to do an implementation of a web server using
a form of asymmetric, multi-process event-driven architecture. A web server
which utilises fibers. It would be nice if you explain the web-server
architecture of vibed in more details.
Keep up with good work!
Regards
Thanks! I will add a page with graphical explanation on the website when
I have some time. But in short all fibers are currently living in the
same thread. Each fiber is registered with one or multiple events inside
of the event driver (libevent based currently). As soon as an event
arrives, the corresponding fiber is resumed. The fiber will then simply
do conceptually this for every blocking operation:
blockingOperation(){
requestOperation();
while( !operationFinished() ) yield();
}
I've thought about sharing fibers across threads but came to the
conclusion that this is probably more trouble than it's worth. An
alternative would be to start multiple event loops in separate threads
and then distribute incoming connections to these loops before starting
a corresponding fiber.
Right now its at least possible to off-load all computational tasks
using std.concurrency or std.parallelism or similar. An efficient
asynchronous wait function for a std.concurrency message or for a
std.parallelism task would be useful though.
Re: Introducing vibe.d!
Am 26.04.2012 23:54, schrieb Trass3r: Looks promising. Though I wouldn't know how to choose between Adam's web framework, Cybershadow's code (seemed like he coded the newsreader in no time and it works very nicely) and yours. Any advice? I guess it just comes down to tast and requirements. If you need high performance in high load situations or if you are used to the router-view-template system (with HAML or Jade templates) vibe should be a good choice. But although I haven't looked very deep into the other two, I know that at least Adam's code contains a lot of features that are not yet available in vibe or are maybe not planned at all (e.g. the handy DOM manipulations). Of course, since vibe is also usable as a simple library, its also possible to combine parts of each framework.
Re: Introducing vibe.d!
The server is back up and I've looked at Flask's blueprints. So they have a more implicit approach with annotations (once D has these, it would be a possible extension for vibe). Right now my corresponding pattern looks like this: // create a global url router auto r = new UrlRouter // create application specific routes r.get(/, showHomePage); // let each sub-component register its routes, possibly in a sub-path registerVibeLog(settings, router); registerTicker(router, /ticker);
Re: Introducing vibe.d!
In _d_throw call abort(). That'll give you a core file. On Apr 26, 2012, at 11:13 PM, Sönke Ludwig slud...@outerproduct.org wrote: Am 27.04.2012 04:19, schrieb Andrei Alexandrescu: On 4/26/12 3:30 PM, Jesse Phillips wrote: On Thursday, 26 April 2012 at 22:05:29 UTC, Robert Clipsham wrote: On 26/04/2012 21:46, Sönke Ludwig wrote: vibe.d This looks awesome! Also on reddit: http://www.reddit.com/r/programming/comments/su9la/vibed_asynchronous_io_that_doesnt_get_in_your_way/ and hacker news: https://news.ycombinator.com/item?id=3896197 Now look what you did, you killed it. Are the Vibe people on this? Though the intent is very nice, this is not looking great right now. Andrei It's back up now and I've made a small watchdog script to avoid this in the future. The problem is that I'm currently using a vibe based reverse proxy instead of nginx and it rarely has an uncaught exception that unfortunately has never been logged with stack trace. I wish I could have made this part rock solid before going live but someone else annouced it before me so it was too late.
Re: Introducing vibe.d!
On 27-04-2012 16:50, Sean Kelly wrote: In _d_throw call abort(). That'll give you a core file. On Apr 26, 2012, at 11:13 PM, Sönke Ludwigslud...@outerproduct.org wrote: Am 27.04.2012 04:19, schrieb Andrei Alexandrescu: On 4/26/12 3:30 PM, Jesse Phillips wrote: On Thursday, 26 April 2012 at 22:05:29 UTC, Robert Clipsham wrote: On 26/04/2012 21:46, Sönke Ludwig wrote: vibe.d This looks awesome! Also on reddit: http://www.reddit.com/r/programming/comments/su9la/vibed_asynchronous_io_that_doesnt_get_in_your_way/ and hacker news: https://news.ycombinator.com/item?id=3896197 Now look what you did, you killed it. Are the Vibe people on this? Though the intent is very nice, this is not looking great right now. Andrei It's back up now and I've made a small watchdog script to avoid this in the future. The problem is that I'm currently using a vibe based reverse proxy instead of nginx and it rarely has an uncaught exception that unfortunately has never been logged with stack trace. I wish I could have made this part rock solid before going live but someone else annouced it before me so it was too late. Why don't we make that some kind of function in druntime? like setAbortOnThrow(bool) or whatever. -- - Alex
Re: Introducing vibe.d!
Sönke Ludwig: The import vibe.d; is actually just a play on the frameworks name and just should be easy to remember. In addition to importing all modules, it also imports a module containing the main function, so this is a special beast. But there is actually a import vibe.vibe; which is the equivalent to the vibe.all. But isn't .all a kind of standard D convention (that is better to generally use)? (I am not sure). Bye, bearophile
Re: Introducing vibe.d!
On Friday, 27 April 2012 at 15:26:02 UTC, bearophile wrote: But isn't .all a kind of standard D convention (that is better to generally use)? (I am not sure). No, not at this point. Some people prefer somepackage._, and there recently has been a proposal by Andrei (DIP16) to allow a special »somepackage.package« module. David
Re: Introducing vibe.d!
On Apr 27, 2012, at 1:27 AM, Sönke Ludwig slud...@outerproduct.org wrote: We still have a more comprehensive benchmark on the table but it seemed to get along happily with about 60MB of RAM usage during a C10k test. The average request time went down to about 6s if I remember correctly. The test was using a dual-core Atom server over a WiFI connection so the results may have been skewed a litte. 6s? Not 6ms? Why so slow?
Re: Introducing vibe.d!
Am 27.04.2012 18:54, schrieb Sean Kelly: On Apr 27, 2012, at 1:27 AM, Sönke Ludwigslud...@outerproduct.org wrote: We still have a more comprehensive benchmark on the table but it seemed to get along happily with about 60MB of RAM usage during a C10k test. The average request time went down to about 6s if I remember correctly. The test was using a dual-core Atom server over a WiFI connection so the results may have been skewed a litte. 6s? Not 6ms? Why so slow? With 1 connections concurrently.. not exactly sure who is reposible but probably either the network or libevent. But some diagnostic benchmarking is in order.
Re: Introducing vibe.d!
Sönke Ludwig wrote:
During the last few months, we have been working on a new
framework for general I/O and especially for building
extremely fast web apps. It combines asynchronous I/O with
core.thread's great fibers to build a convenient, blocking
API which can handle insane amounts of connections due to
the low memory and computational overhead.
Some of its key fatures are:
...
vibe.d looks great! Looking through the docs, it feels clean and
hip like node.js
However, there are a couple refactoring choices I think would
have made it even more hip, consider the home page example:
import vibe.d;
void handleRequest(HttpServerRequest req,
HttpServerResponse res)
{
res.writeBody(Hello, World!, text/plain);
}
static this()
{
auto settings = new HttpServerSettings;
settings.port = 8080;
listenHttp(settings, handleRequest);
}
vibe.d as the project name is great, but why have module vibe.d
and not simply vibe? Or, why prefix all the types with
HttpServer when you could separate those objects into a new
module and drop the prefix. I think those simple changes would
make an already polished library shine a bit brighter:
import vibe.http;
void handleRequest(Request req, Response res) {
res.writeBody(Hellow, World!, text/plain);
}
static this() {
auto settings = new Settings;
settings.port = 8080;
listen(settings, handleRequest);
}
the echo server would look like:
import vibe.tcp;
static this()
{
listen(7, (conn){ conn.write(conn) });
}
Also, it's great to see MongoDB support built in :D, but isn't
there a way to trim down the query object from:
BSON({name: BSON(Peter)})
to:
BSON({name:Peter})
by writing a generic AssosiativeArrayToBSON function? That way
you could overload MongoDB's find/save/etc functions to accept
arbitrary AAs as queries, which would end up looking a lot
slicker in the docs I think. Plus, through CTFE it's possible
this conversion could happen at both compile-time and run-time:
auto db = new MongoDB;
auto col = db[test.collection];
foreach (doc; col.ctFind!([name:Peter])) {
...
}
Just some ideas in case you haven't thought of them already :)
Overall vibe.d looks like a fantastic library. Thank you and
Congrats!
Re: Introducing vibe.d!
On Friday, 27 April 2012 at 19:40:53 UTC, F i L wrote: vibe.d as the project name is great, but why have module vibe.d and not simply vibe? Or, why prefix all the types with More complicated projects/frameworks ontop of vibe which implements it's own main is free to use vibe.all. But I think... import vibe.d; ... makes perfect sense because it contains main()!
Re: Introducing vibe.d!
Le 26/04/2012 22:46, Sönke Ludwig a écrit : During the last few months, we have been working on a new framework for general I/O and especially for building extremely fast web apps. It combines asynchronous I/O with core.thread's great fibers to build a convenient, blocking API which can handle insane amounts of connections due to the low memory and computational overhead. Some of its key fatures are: - Very fast but no endless callback chains as in node.js and similar frameworks - Concise API that tries to be as efficient and intuitive as possible - Built-in HTTP server and client with support for HTTPS, chunked and compressed transfers, keep-alive connections, Apache-style logging, a reverse-proxy, url routing and more - Jade based HTML/XML template system with compile-time code generation for the fastest dynamic page generation times possible - Built-in support for MongoDB and Redis databases - WebSocket support - Natural Json and Bson handling - A package manager for seemless use of extension libraries See http://vibed.org/ for more information and some example applications (there are some things in the works such as an etherpad clone and an NNTP server). vibe.d is in a working state and enters its first beta-phase now to stabilize the current feature set. After that, a small list of additional features is planned before the 1.0 release. The framework can be downloaded or GIT cloned from http://vibed.org/ and is distributed under the terms of the MIT license. Note that the website including the blog is fully written in vibe and provides the first stress test for the implementation. Regards, Sönke It is awesome. I think some part of this could be integrated into phobos after some refactoring. I think about databases for instance, or some http libraries.
Re: Introducing vibe.d!
On 4/27/12 1:42 PM, deadalnix wrote: It is awesome. I think some part of this could be integrated into phobos after some refactoring. I think about databases for instance, or some http libraries. This is an ignorant question, but why not the async IO stuff? It seems like it could be widely useful. Or is it only designed for this kind of use?
Re: Goldie Parsing System v0.9 - Tools
Nick Sabalausky wrote: Goldie is a series of open-source parsing tools, including an optional D programming language library called GoldieLib. Goldie is compatible with GOLD Parser Builder and can be used either together with it, or as an alternative to it. Overview of changes in v0.9: - Added support for DMD 2.059. - Dropped support for DMD 2.054 and below. - GoldieLib: Renamed Language.loadCGT to Language.load. - GRMC: Grammar Compiler: Major bugs fixed (see changelog link below) - New tool: AlterCGT. - Command line options for all tools are now processed with getopt and use standard Unix conventions (even on Windows). This means what used to be -foo:bar or /foo:bar must now be written as --foo=bar or --foo bar. (But /? is still supported as an alternative to --help.) - Misc changes/updates to various tools (see changelog link below) Full ChangeLog: http://www.semitwist.com/goldie/ChangeLog/ Goldie Homepage: http://www.semitwist.com/goldie GoldieLib is pretty nice. :) -- Dejan Lekic - http://dejan.lekic.org
Re: Introducing vibe.d!
On Friday, 27 April 2012 at 20:15:05 UTC, Tove wrote: On Friday, 27 April 2012 at 19:40:53 UTC, F i L wrote: vibe.d as the project name is great, but why have module vibe.d and not simply vibe? Or, why prefix all the types with More complicated projects/frameworks ontop of vibe which implements it's own main is free to use vibe.all. But I think... import vibe.d; ... makes perfect sense because it contains main()! Ah I see, I was thinking the example wasn't the full code. Still, I think doing something like: import vibe, vibe.html; where vibe has main() and vibe.html has html is still better semantics.
Re: Introducing vibe.d!
On 4/27/12 4:46 AM, Sönke Ludwig wrote: During the last few months, we have been working on a new framework for general I/O and especially for building extremely fast web apps. It combines asynchronous I/O with core.thread's great fibers to build a convenient, blocking API which can handle insane amounts of connections due to the low memory and computational overhead. Some of its key fatures are: Impressive. The website also looks really nice, and it's very fast. I'll definitely play with it and slowly try to make it into my workplace, hehe.
Re: Introducing vibe.d!
On 4/27/12 2:50 PM, Brad Anderson wrote: On Thursday, 26 April 2012 at 20:46:41 UTC, Sönke Ludwig wrote: During the last few months, we have been working on a new framework for general I/O and especially for building extremely fast web apps. It combines asynchronous I/O with core.thread's great fibers to build a convenient, blocking API which can handle insane amounts of connections due to the low memory and computational overhead. Some of its key fatures are: - Very fast but no endless callback chains as in node.js and similar frameworks - Concise API that tries to be as efficient and intuitive as possible - Built-in HTTP server and client with support for HTTPS, chunked and compressed transfers, keep-alive connections, Apache-style logging, a reverse-proxy, url routing and more - Jade based HTML/XML template system with compile-time code generation for the fastest dynamic page generation times possible - Built-in support for MongoDB and Redis databases - WebSocket support - Natural Json and Bson handling - A package manager for seemless use of extension libraries See http://vibed.org/ for more information and some example applications (there are some things in the works such as an etherpad clone and an NNTP server). vibe.d is in a working state and enters its first beta-phase now to stabilize the current feature set. After that, a small list of additional features is planned before the 1.0 release. The framework can be downloaded or GIT cloned from http://vibed.org/ and is distributed under the terms of the MIT license. Note that the website including the blog is fully written in vibe and provides the first stress test for the implementation. Regards, Sönke I had to copy the included .lib files into bin in order to build the examples but so far, so good. This is awesome. How did you install it? I can't find the install.sh script anywhere...
Re: Introducing vibe.d!
On 4/28/12 8:12 AM, Ary Manzana wrote: On 4/27/12 4:46 AM, Sönke Ludwig wrote: During the last few months, we have been working on a new framework for general I/O and especially for building extremely fast web apps. It combines asynchronous I/O with core.thread's great fibers to build a convenient, blocking API which can handle insane amounts of connections due to the low memory and computational overhead. Some of its key fatures are: Impressive. The website also looks really nice, and it's very fast. I'll definitely play with it and slowly try to make it into my workplace, hehe. How to use it? ./bin/vibe usage: dirname path sh: /vpm.d.deps: Permission denied Failed: 'dmd' '-g' '-w' '-property' '-I/../source' '-L-levent' '-L-levent_openssl' '-L-lssl' '-L-lcrypto' '-Jviews' '-Isource' '-v' '-o-' '/vpm.d' '-I/' Error: cannot read file source/app.d Failed: 'dmd' '-g' '-w' '-property' '-I/../source' '-L-levent' '-L-levent_openssl' '-L-lssl' '-L-lcrypto' '-Jviews' '-Isource' '-v' '-o-' 'source/app.d' '-Isource' I also can't find the install.sh script...
