Hi Poul-Henning! On Tue, May 06, 2025 at 01:13:22PM +0000, Poul-Henning Kamp wrote: > This started as a private conversation, but I think it belongs > on varnish-dev now: > > I'm not really sure I know what the "previous plan" was any more, but > I want this sorted out for 8.0. > > My conversation with dridi@ yesterday revealed that we may have > misunderstood each other somewhat, as there are two different issues > involving shared libraries. > > The first issue, let us call it "vtest+lib" is to make vtest a > package which installs an include, static and/or dynamic libraries, > and a binary which uses the shlib, so that the varnish-cache project, > and possibly haproxy, can link our specific test-binary against the > library of choice. > > This only really makes sense if we go the full monty with the vtest > repos: Publishing releases, making packages, pushing them into > distros etc. The alternative is a simple source dependency, using > some variant of git submodules or shell-scripts to do the dirty > deed. > > The second issue is that vtest uses a lot of stuff copied from > varnish-cache's lib directory. > > IMO that is just an annoyance. > > To fix it "properly", the vtest repo could pull that stuff out of > the varnish-cache repo, but that would pretty much defeat the entire > reason why we created the vtest repo to begin with. > > Alternatively, we could make a third repos containing "varnish-lib" > which then both varnish-cache and vtest repos could depend on, as > well as any other repos which have copied stuff from there over the > years. > > There is some good stuff in varnish-cache/lib, but personally I > do not feel it is enough for us to start yet another repository. > > If we do create a varnish-lib repos, then the next question again > becomes "package or source dependency". > > (I thought what I nixed at some previous occation was the "varnish-lib" > idea, but dridi@ thought it was the "vtest+lib" idea.) > > Finally there is an alternative where we do "vtest+lib", let the > vtest repos take over responsibility for the lib stuff and have > varnish-cache repos get it from there. That would add no further > dependencies, eliminate the duplication of the lib-stuff, and > give any third parties a reasonably sized package to depend on > if they want to use the lib-stuff. > > So to me the core question seems to be: Are we willing/going to > make vtest+lib packages ? > > And of course, since HAproxy also uses vtest, it is not something > we alone decide... (Willy cc'ed)
Hmmm I'm seeing that you're facing the same issues that everyone faces when starting to reuse code. IMHO, going the "varnish-lib" way is going to be annoying. I've experienced the same with some of my personal projects that are used in haproxy (libslz, ebtree etc): in practice, haproxy is almost the only user, which means that bugs are most likely to be detected there, and missing features as well. I hesitated between adding an external dependency and integrating the code. I started with the external dep and it was too painful (even for end users), so I ended up integrating a copy of the libs (they're tiny) in the haproxy project. So this means that haproxy contributors willing to patch certain areas are sometimes confused about how to proceed: patch directly into haproxy or send the patch to the upstream lib and hope that it will flow down. But I generally have no problem supporting bidirectional patching, because such code changes very slowly, so if I have two patches to port or backport every year, that's no big deal. Still it's something you need to keep in mind. In your case, I sense that this "varnish-lib" would be the library providing about everything. I think you'd face a huge overhead by maintaining it out of the project, for little gains. What *could* possibly work would be to make it easy to extract such code parts from varnish into an external varnish-lib project that is only updated when needed. Then vtest would depend on that. This would mean that you do have a copy, but not a copy burried into vtest, instead a copy that constitutes an autonomous project. It's much easier to keep up to date, and its sole purpose is to permit other tools to continue to work. So basically a full export from time to time does the job *if needed*. And it's not *that* unusual. For example in the linux kernel, one can to "make install-headers" and get a full directory of all the system headers, that is consumed by whatever libc you use. In any case I don't think that varnish using vtest as an external dependency would be a good idea. It would first create an extra maintenance burden for varnish, and second, create difficulties when other projects start to adopt vtest because they will not realize that the changes they propose can have an impact on varnish itself. Just my two cents, Willy _______________________________________________ varnish-dev mailing list varnish-dev@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
