Hey, I'm curious what you are trying to measure in regards to Clojure. As far as I know there is not a single Web Server actually written in Clojure, they are all written in Java and we just use them. The Compojure benchmark you linked uses the default Jetty Adapter for Ring (like Rack in Ruby land). http-kit is written in Java as well with only tiny bits of Clojure.
So to answer your question: If I wanted to write something that satisfies your requirements (and only those) I would probably take something like undertow and just use that. See https://github.com/thheller/undertow-test https://github.com/thheller/undertow-test/blob/master/src/undertow_test/core.clj There are some tweaked settings in there which I used from the Java version of the techempower benchmark, I'm doubt they are ideal for your benchmark but I cannot run a test like you describe on my machine to find better parameters. OSX just runs out of sockets after a couple seconds. You'll need leiningen [1] to produce the "uberjar". lein uberjar java -cp target/undertow-test-0.1.0-SNAPSHOT-standalone.jar clojure.main -m undertow-test.core localhost 5500 That launches a JVM with the example server bound to localhost:5500. I have not used undertow before myself and don't know about recommended GC parameters. The Clojure bits do very little and you are basically just testing undertow at this point so I'd imagine the performance to be identical to a similar pure Java version. I doubt that this is what you have in mind when trying to produce a "Clojure benchmark" but since all we Clojure-Folk do is built upon Java servers anyways and I wouldn't want to make it look bad by using things we don't really need (ring, compojure, etc ...). I don't use ring or compojure myself so I can't say how much overhead they would introduce. I'm happy to help if you have any Clojure related questions for your benchmark. If all you really want to test is the web server performance it probably doesn't make much sense to include Clojure since there are no Clojure web servers (AFAIK). HTH, /thomas [1] http://leiningen.org/ On Wednesday, April 22, 2015 at 3:05:29 PM UTC+2, Jesper Louis Andersen wrote: > > Hi, > > I'm trying to build up a different kind of web server framework benchmark, > where measurement is not on peak performance, but on capacity and latency. > That is, the numbers we keep stable are: > > * 10k connections > * 30k req/s > * 2053 bytes of static content served by a GET request on a simple route > * Near perfect 1 gigabit connection between two machines > * The test runs for 20 minutes. > > This is in contrast to many such measurements. The recent "round 10" > TechEmpower benchmark will for instance let connections vary, try to push > as many req/s through and report the maximal number for a "Hello World!" > message. So for one system, they will report the number for 1024 > connections, whereas another system will have 4096 connections, whichever > has the highest peak performance. This is a big no-no in my book. > Furthermore, I'm questioning the correctness of their latency measurements > because they report average latency and std. deviation without establishing > that their distributions are normal. And they don't report the median > latency so we can see they aren't. > > The thing I'm interested in is latency of the 30k req/s, run over a 20 > minute period. That is, employing Gil Tene's HDRHistogram, we can > accurately measure latencies for each request and make a distribution > function plot of those latencies. I'm sitting with approximately 15 > frameworks, spread out over multitiple languages. The numbers are vastly > different from typical benchmarks and they are interesting to report. > > For Clojure, I started out with a simple adaptation of the Compojure > benchmark from the TechEmpower-land. But it exhibits lots of queueing when > trying to process requests simply due to maximizing the CPU cores of the > System-under-test. My JVM-fu is quite weak, which is a big problem in these > tests. I more or less need a setup, which has some kind of well-tunedness > in advance, because I have little hope in achieving it myself on my own. > It's somewhat like 10 years ago I last tuned GC on a JVM and I bet things > change. Currently Clojure is in the ballpark of Python and Ruby in speed, > which is so slow I'm going to argue that there is something wrong with my > test implementation: > > http://imgur.com/wCFnFnd > > (X-axis are percentiles, note it's compression toward the 0th percentile. > I'm interested in the upper percentiles) > > The graphs of Python, Ruby and Clojure/Compojure here are clear indicators > that queueing/stalling is going on even at the very low percentiles. The > solutions Go and Java/Undertow are in the graph for comparison with a > couple of frameworks which does not exhibit queueing[0]. > > The highest scoring benchmark in the techempower rounds is a combination > of Clojure and Resin (whatever resin is, I've not really dug too much into > it), but I'm told that http-kit may be a more viable route. Also, I might > be needing some underlying web server I guess, and what are good > suggestions there?. > > So my question is this: "If you were to write a webserver where curl > http://localhost:8080/ would return the first 2053 bytes of "Alice in > Wonderland"[1], what would you do?" In particular if the above data is > known: capacity is 10k connections, rates are 30k req/s. Goal is > low-latency operation. As I said, the current numbers are just too far off, > but in contrast to Ruby/Python, I think Clojure might just be my bad > implementation of what is going on. That is, if you went tabula rasa on my > frankenstein monster, what would you do? > > [0] What is not present on this graph is the bi-modal behavior of Go and > Undertow once GC kicks in, however. > [1] This is a grave mistake! It should have been "Jabberwocky!" > > The techempower code is here: > https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Clojure/compojure > > and my field-surgery is below for hello.clj: > > (ns hello.handler > (:import com.mchange.v2.c3p0.ComboPooledDataSource) > (:use compojure.core > ring.middleware.json > ring.util.response) > (:require [compojure.handler :as handler] > [compojure.route :as route])) > > (def plaintext > "Test 6: Plaintext" > {:status 200 > :headers {"Content-Type" "text/plain; charset=utf-8"} > :body "CHAPTER I. Down the Rabbit-Hole Alice was beginning to get very > tired of sitting by her sister on the bank, and of having nothing to do: > once or twice she had peeped into the book her sister was reading, but it > had no pictures or conversations in it, <and what is the use of a book,> > thought Alice <without pictures or conversations?> So she was considering > in her own mind (as well as she could, for the hot day made her feel very > sleepy and stupid), whether the pleasure of making a daisy-chain would be > worth the trouble of getting up and picking the daisies, when suddenly a > White Rabbit with pink eyes ran close by her. There was nothing so very > remarkable in that; nor did Alice think it so very much out of the way to > hear the Rabbit say to itself, <Oh dear! Oh dear! I shall be late!> (when > she thought it over afterwards, it occurred to her that she ought to have > wondered at this, but at the time it all seemed quite natural); but when > the Rabbit actually took a watch out of its waistcoat-pocket, and looked at > it, and then hurried on, Alice started to her feet, for it flashed across > her mind that she had never before seen a rabbit with either a > waistcoat-pocket, or a watch to take out of it, and burning with curiosity, > she ran across the field after it, and fortunately was just in time to see > it pop down a large rabbit-hole under the hedge. In another moment down > went Alice after it, never once considering how in the world she was to get > out again. The rabbit-hole went straight on like a tunnel for some way, and > then dipped suddenly down, so suddenly that Alice had not a moment to think > about stopping herself before she found herself falling down a very deep > well. Either the well was very deep, or she fell very slowly, for she had > plenty of time as she went down to look about her and to wonder what was > going to happen next. First, she tried to look down and make out what she > was coming to, but it was too dark to see anything; then she looked at the > sides of the well, and noticed that they were filled with cupboards......"}) > > > (defroutes app-routes > (GET "/" [] plaintext) > (route/not-found "Not Found")) > > (def app > "Format responses as JSON" > (wrap-json-response app-routes)) > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
