VDD 2022q1 NOTES Paris 2022-04-21 - 2022-04-22
#1 martin on varnishlog scalability - logging is a killer feature which users' business cases depends upon - Some users would rather see clients wait than losing a log record. - log clients are single threaded - for multiple queries, basic structure building done per log client - ideas - binary log nice to have, not a solution - solve desync issue by giving the option to report lost entries - or throttling / blocking varnishd - merge logging into varnishd - drop session logging, copy session attributes into request scope - potential to optimize logging away (kills varnishlog -d, could keep internal buffers longer instead) - varnishlogd muti-query multi-thread (read shm and fan out to pipes/udses) - structured shared memory - use utility threads inside varnishd to write logs to fs, which is then read by the clients. - compiled queries - pluggable, custom log implementations in varnishd, renovate interface first (phk) - side topics - custom tags for vmods - log all counter diffs #2 phk jumps agenda and talks about how he spent the covid period (rational R1000 machine at datamuseum.dk) #3 Varnish Association / Charity / Forening / Verein / Foundation - martin talks about VS efforts in this direction - VS owners have welcomed the idea - slide set from law firm LUCENTUM - disputed: Transfer copyright to foundation? - maybe not even relevant? - Lots of commitees - trademark: VS AB owns, licenses it to Verein - cost: VS willing to take on running costs - Should the entity spread money for development "centralized VML" - nils presents ideas for bylaws - most important questions - do the bylaws get the verein involved in money etc? - "grassroots" or "corporate" style? #4 TLS - arguments: - ToS / QoS by content - Adaptive Congestion Control by content - kernel TLS for mallanox hw offload - use cases - dynamic loading of certificates based on SNI from VCL - not just by name, want to prefer specific cert - session key sharing - vcl_${proto} {} - client certificates for backend connections - should use something along the lines of CF keyless - no private keys in varnishd worker process - cert server should be pluggable, should have some kind of CLI - could significantly complicate H3 - martin: have done poc for plugging "keyless" as openssl encryption engine - seems hard to do it ssl lib implemention independent - try to not make it hard to switch impl - VS: linking against stock openssl - stack transports? - do the work - UPLEX has sponsor - VS might upstream as open source - limit the scope to OpenSSL >= 1.1 ? - strawman the vcertd (dridi?) - varnishtest - dridi has done 2 rounds of polishing of https in varnishtest - dridi has plans for a 3 round before porting to trunk - to submodule SOONER (revive slinks PR?) - when varnish is branched off, the vtest code can be bundled (no submodule) - needs patch back-ports convenience - refs https://github.com/vtest/VTest/pull/2 https://github.com/varnishcache/varnish-cache/pull/2838 #5 H3 - Lars is still working on the quant project - should look at msquic - quictls openssl fork for quic - haproxy has own quic - argument against openssl is that h3/quic moves fast - probably needs a new survey of available implementations (asad) #6 extensions, pluggable stevedores and protocols (xx) - phk talks about changes which are going to happen - martin brings up loading extensions in mgmt, seems not to be a killer argument - built-in stevedores stay in varnishd - built-in stevedores will register before extensions, so default=malloc|umem will remain - martins batch insertion - HSH_Insert() EXP_Insert() BAN_Insert() batched functions - martin+slink #7 >1 level of return(vcl, prefix="example.com"") - current implementation is just defensive - parameter with absolute limit on switches, default should remain just 1 level - structure label names? xyz/abc -> xyz can only return (vcl(xyz/abc)) - is the prefix argument absolute or relative - write the docs, implement #8 bundled VMODs - discussion about varnish-modules and var vs. objvar vs. native variables in vcl objvar today: vcl_init { new myvar = taskvar.string(); } myvar.set("foo"); set resp.http.foo = myvar.get() objvar with designated setter/getter methods: myvar = "foo"; set resp.http.foo = myvar; native scoped variable suggestion: {vcl,client,backend}.* symbols set client.var.foo = <string expr>; set backend.var.foo = <string expr>; set vcl.var.foo = <string expr>; native scoped variable alternate suggestion: {be{re{q,sp}}}.var.* with req=>bereq and {beresp,obj}=>resp copies another scoped variable suggestion: [global] {[req|bereq|resp|beresp]} [type] var; beresp variables would be persisted with the object -> write a concrete proposal for user docs (guillaume) xkey: - vmod as is only works with non-persistent storage - should look at http-wg suggestion - important feature suggestion for inclusion into varnish-cache accept: Filter accept-like headers bodyaccess: Client request body access header: Modify and change complex HTTP headers VS to upstream headerplus? str: String operations should turn into type methods, need arguments for type methods task for dridis apprentice tcp: TCP connections tweaking vsthrottle: Request and bandwidth throttling should be redone with modules and better methods side discussion: issues with vmod cookie - removes duplicate cookies - need iterator over multiple cookies -> simon please open cookie #9 compiled VCL sideloading - guillaume: still need vcl even if overriding cc_command for vmod import - we should put vmods into .so - complicated because of vmod random names #10 slash stevedores (fixed size mem + persistent tiered) #11 Homework assignment for friday, read: https://httpwg.org/http-extensions/ - asad recommends wg meeting videos, https://www.youtube.com/user/ietf/videos #12 synth bodies and filters - phk wants to remove the option to reset the synth body - vsb is contigous region, good case for discontinous object creation - dridi is of the opinion that vcl resetting the beresp.body should pay the cost of resetting a vfp #13 via backends https://docs.varnish-software.com/varnish-cache-plus/features/backend-ssl/ dridi syntax suggestion: backend proxy { .type = connect; # registered by vext proxy .host = "..."; } backend b{1..3} { .type = http1; # built-in, default value .tls = true; .client_cert = "/bla/c{1..3}"; .via = proxy; .host = "..."; .authority = "..."; } backend default { .type = round_robin; # registered by vext directors .via = [b1, b2, b3] } #14 multiple headers with the same name shall implement - xtaskvar.foreach_header(HEADER, SUB) - xtaskvar.foreach_cookie(HEADER, SUB) - HEADER.count - std.sanitize_headers() - collect all collectable headers - C code fail for more duplicate headers - date? Legacy headers which are SF compatible could become binary - https://mnot.github.io/I-D/draft-nottingham-binary-structured-headers.html - https://www.rfc-editor.org/internet-drafts/draft-ietf-httpbis-retrofit-00.html -> how could VCL look if those became a reality? #15 parallel/async CLI Implement cli-like interface to vkeyservd with async/parallel syntax Check how this could look like for varnishd Only specific commands would become async (vcl.load and vcl.inline mostly) bans could serve as a template (vcl.load -a name ... -> 205 async started / vcl.list shows "pending"/"failed") buffer output in vsb, cli command to query (vcl.show -o vcl_name, -o for compiler output) martin suggests a "job interface" jobs.list / jobs.wait name #16 optional vmods / conditional code? import cookie [from PATH] [or skip]; with (import cookieplus) { # cookieplus.do_stuff() } else with (import cookie) { # cookie.do_stuff() } else { # regex time } -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstraße 32 22301 Hamburg tel +49 40 28805731 mob +49 170 2723133 fax +49 40 42949753 xmpp://sl...@jabber.int.uplex.de/ http://uplex.de/
signature.asc
Description: OpenPGP digital signature
_______________________________________________ varnish-dev mailing list varnish-dev@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev