Hi all, So, almost 2 months after the initial announcement and quite a lot of dogfooding, varnish-rs has seen a few releases and grown a bit. Even though I still consider it in the alpha stage, I wanted to present a couple of vmods I built on top of it.
The first vmod is the smallest one: vmod_rers ( https://github.com/gquintard/vmod_rers) that deals with Regular Expressions in RuSt. A cursory look at the VCC file ( https://github.com/gquintard/vmod_rers/blob/main/vmod.vcc) will quickly reveal that it does way less than ` https://code.uplex.de/uplex-varnish/libvmod-re2/` however, there's still a couple of nice things about it: - regex are cached in a LRU store, so you don't have to recompile the same dynamic expression over and over again - named patterns! - there is both a VDP and a VFP, in other words: you can modify content as it enters the cache AND as it leaves it. Note however that to do this, we do cache the whole body, use this feature responsibly. The second one is beefier: vmod_reqwest ( https://github.com/gquintard/vmod_reqwest) is a vmod_curl on steroids. The initial idea here is also to allow the VCL to send and receive HTTP requests, but things got a bit out of hand, and in the end, it has a few extra features: - handle multiple requests at the same time, within the same task - fire-and-forget, non-blocking mode: you can send 15 requests at the same time and continue regular VCL processing as they are in flight - proper VCL backend, instead of the regular static backends, you can use dynamic ones backed by reqwest, offering: - redirect following, if the response sports a 30X status, the vmod can automatically follow it - connection pooling, of course - HTTPS backends - proxies for both HTTP and HTTPS - automatic brotli (and gzip and deflate) decompression There are a few things missing at the moment, notably probes, but they are coming. So, things look pretty exciting BUT I haven't tested that in production, and I've mostly been operating in a vacuum so far, so I would love some feedback on APIs, performance, bugs, etc. -- Guillaume Quintard ---------- Forwarded message --------- From: Guillaume Quintard <[email protected]> Date: Sun, Dec 5, 2021 at 5:32 PM Subject: Writing vmods in Rust To: [email protected] <[email protected]> Hi everyone, I've been working for a while on a little project that ended up taking quite a chunk of my time, and while it's nowhere near done, I feel like it's time to show the current progress. *In other words: that stuff ain't ready, don't put it in prod.* Here we go: we can *build vmods using pure rust*. For example, this guys: https://github.com/gquintard/varnish-rs/tree/main/examples And the crates `varnish` and `varnish-sys`, that make this possible have been published on crates.io, and we have documentation! https://docs.rs/varnish/latest/varnish/ Without trying to start a language war, there are a few benefits to using Rust over C: - the language is designed to prevent a few classes of annoying bugs, and the compiler is very good at explaining where you went wrong - a lot of the C practices (assert everywhere, miniobj, etc.) are basically baked into Rust - the barrier between C and Rust is razor-thin, with almost zero overhead cost (curse you, null-terminated strings!) - the tooling is absolutely stellar - string manipulation is sane (yeah, easy stab at C, but I had to do it) What you get at the moment: - few dependencies: you only need cargo, python3 and the libvarnish dev files to get going - write the same vmod.vcc as you would for C, the boilerplate is generated for you - building is literally just one command: `cargo build` - automatic type translation so you can write pure rust, and never need to see the `VCL` objects under their C form ( https://docs.rs/varnish/latest/varnish/vcl/convert/index.html) - support for PRIVobjects ( https://github.com/gquintard/varnish-rs/blob/main/examples/vmod_object/vmod.vcc and https://github.com/gquintard/varnish-rs/blob/main/examples/vmod_object/src/lib.rs ) - you can iterate through and modify the HTTP objects easily ( https://docs.rs/varnish/latest/varnish/vcl/ctx/struct.Ctx.html) What you don't get for now: - a comprehensive API, some bits are still missing, like WS_ReserveAll/WS_Release, but because of the type conversion, you can possibly live without it - a stable API. I'm heavily working on it, and things will change twenty times before the next Varnish release (but it doesn't matter as the crates are version-locked) - something based solely on the public library (libvarnishapi), I reimplemented some stuff using private bits to avoid copies - all the types. I only implemented a subset of VCL types, but fret not, the others will come. - a sensible boilerplate code generation, the current approach involves embedding a python script that imports `vmodtool.py` and running it from rust. But there's a plan to fix that In case you want to try it, I've also created a minimal example repo that you can play with: git clone https://github.com/gquintard/vmod-rs-template.git cd vmod-rs-template cargo build cargo test I will continue working on expanding the API, documentation and will start filing issues to come up with a roadmap. If you are interested in helping, have questions or feedback, please do ping me, I'd love to hear from you. Cheers! -- Guillaume Quintard
_______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
