-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Leif,
Thanks for answer! What you write about is what we tried at first, using Tornado. It works, except it simply does not have performance good enough: 600 req/sec while we need 30,000 req/sec (yes 30 thousand). This is for entire subsystem which can consist of more than 1 machine but at Tornado performance that would be at least 50 cores over many serves + HA redundancy + all the associated overhead... Currently we're into node.js which is able to serve 4000-5000 req/sec, still not quite good enough. Varnish performance is so high that it would be very attractive in this role. Re application logic in VCL I don't think we'd suffer this much: it's "just" about fast cache. About the only serious problem I could see is if failover and load balancing in varnish can't be made work well enough (detect failure quickly and start using spare/backup systematically instead of timing out on failed backends, cause uneven load on backends, etc). Thanks! MK W dniu 7/30/2013 14:46, Leif Pedersen pisze: > Hi, > > I'm not a varnish dev but I've been working with varnish and DBs > for a long time. > > My knee jerk is that you'll end up with more application logic in > vcl than vcl is suitable for. Vcl can't loop or touch response > bodies (without vmods). It'd be kind of neat to skip middleware and > plug varnish straight into the db, but I wouldn't guess it to be > worth the effort. > > I'd instead write a mini web server using uwsgi or CherryPy to act > as middleware. These tools have great memory footprints and > performance. > > - Leif > > On 2013-07-30 6:30 AM, "Marcin Krol" <[email protected] > <mailto:[email protected]>> wrote: > > Hello, > > This is a peculiar topic that I think goes beyond typical use of > varnish so I post it here. > > At my company we have a need peculiar sort of infrastructural > subsystem: > > - HTTP requests are done to find if smth is cached - RDBMS (mysql, > oracle) backends - Other subsystems as backends > > Clients use unified protocol based on (simple) http requests to > get their data. (it's for this reason that we do not use caches > built into rdbms directly, as well as we do not want to do tight > coupling of a particular client to a particular rdbms or > subsystem) > > > Either we write the whole thing ourselves or we use smth else like > varnish. > > > I like the thought of using varnish, although I'm not sure if it > this is not shoehorning it into such role. However, when it comes > to caching, load balancing, failover and cached HTTP results > serving it's ideal in such role. > > The only problem is backend. Essentially, what we need is e.g. for > mysql backend: > > on cache miss: > > - connect to mysql > > - run the query we received in GET/POST/whatever > > - JSONIFy result (query results are not big in our application, > limited size of the result is a tolerable limitation for us) > > - cache result, return it > > > on cache hit: > > - retrieve from cache, return it > > > (and so on for other backends) > > > Is this feasible? Is it even sane? Should I use smth else maybe? > > Essentially, what we need are pluggable, modular backends. > (obviously we can handle writing the part that transforms > particular backend response into HTTP response, the snag is how to > plug this correctly into backend usable by varnish) > > I was thinking about using VMODs but none of the modules available > seem to meddle with backends themselves somehow. > > > Thanks! MK > > > _______________________________________________ varnish-dev mailing > list [email protected] > <mailto:[email protected]> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJR97uhAAoJEFMgHzhQQ7hOSPkIAIRCFwLmt5FtIa/VHVJoMxZR BeLx5yKZBSCHJPhBIeywK4sd7+bW7AMLAEh9VYRfe5c7yuZO8mxAXYmvVXPmD/cl ySUqZJD0LnpdL1kos25K/r8vn6PRPul2jm22u0SloKqQ5ME8TGNsk/SLmFbiLwCt nhfL7ia+19/ZVK3XSZ5pcvsCyrM8flFS3TdM5TeKWDBhxu0uuccaBkuYy0qOK0zs 5YoLUD9GRwj9WN0pt2xFWYncgFP2wGsPzO9YrpVyc73ExXviI1LF+i+1K2wrOk+v nYqekDAHufQmLfeupY5Dq/EHyWnN2Yec/cLaiUQADZKs7ua43GkiUqqJ/pxIUbk= =hymW -----END PGP SIGNATURE----- _______________________________________________ varnish-dev mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
