Re: [squid-dev] RFC submodule repositories
On 8/1/22 22:32, Amos Jeffries wrote:
On 1/08/22 03:09, Alex Rousskov wrote:
On 7/31/22 00:29, Amos Jeffries wrote:
When PR #937 merges we will have the ability to shuffle old helpers
into a separate repository that users can import as a submodule to
build with their Squid as-needed.
What will we optimize by introducing such a significant
complication as git submodules?
( So every change to Squid has to be justified as an optimization now.
Right... )
The verb "optimize" is generally defined as "to make the best or most
efficient use". In computing, that definition is relaxed to mean any
improvement (not necessarily the "best"). Both definitions would work in
this context, but I used "optimize" in the latter ("to improve") sense.
I hope we can all agree that Squid changes should improve Squid.
We "optimize" future code maintenance workload with the ability to drop
outdated helpers which are still required by a small group of users but
no longer want to be actively developed by the core dev team.
N.B. Unless something else changes, simply moving "dropped" helper code
to a submodule would require reviewing dropped helper update PRs in an
aggregate "take it or leave it" mode. No dropped helper code change
requests would be appropriate for such "updated dropped helper H to
version V" pull requests. I do not know whether that will improve core
developer lives, but it is likely to require making more judgement calls
than we have to make today, and jusdgement calls can be especially
controversial/painful.
How would moving "dropped" helper code to a submodule help core
developers? AFAICT, just moving a dropped helper into a submodule
creates more work for core developers because they now have one more
thing to worry about. Git submodules (and git in general!) have very
little impact on making "./bootstrap.sh && ./configure && make && sudo
make install" produce the desired outcome -- with or without submodules,
we still need to make the whole thing build and work correctly. Thus,
any improvements must be rooted in some other, yet unspecified Project
changes besides moving some helper code into submodules. What other
changes will make that extra core developer work worth it?
In other words, it is possible that the RFC associates "git submodules"
with some unspecified changes outside the Squid git repository
structure[1] that RFC readers cannot guess. Identifying any important
changes the RFC associates with git submodules may help us make progress.
[1] For example, one common submodule use case is having different
users/permissions for different parts of the Project -- essentially
having multiple core developer teams with varying levels of access to
critical Project resources, but all collaborating under one Project
roof. It is often impractical to achieve that kind of access controls
within one git/GitHub repository. I am not saying this example applies
to this RFC -- I am just illustrating how submodules may facilitate
important changes outside of low-level git mechanisms themselves.
Case in point being CERN who still require our bundled LanManager
helper(s) for some internal needs. That requires a lot of deprecated and
rarely touched code being maintained for only one (albeit important)
user case.
That code could all be shuffled to a separate repository outside the
official Squid release, but maintained by developers that support CERN
needs.
Sure, but we clearly do not need git submodules to "shuffle LanManager
helper(s) code to a separate repository". Why submodules? The RFC
implies, without elaborating, that submodules are the right solution to
some problem. Several RFC reviewers correctly point out that submodules
are associated with significant "cons". I am asking what the "pros" are.
It is a very reasonable request IMO.
Agreeing on the _problem_ definition may also help shorten the number of
iterations required to reach consensus. What do we really mean when we
say that "dropped" helpers are "not actively developed by core
developers"? To me, the lack of core developer involvement implies that
PRs against Squid master may break "dropped" helpers and that "dropped"
helper needs are assigned much lower weight compared to "native" Squid
improvement needs, but others are likely to interpret that phrase
differently.
What (if any) updates do we need to make to Anubis and other
infrastructure so support git submodules ?
That answer probably depends, in part, on what support guarantees we
are going to issue for those submodules and on what our submodule
update policy will be.
IMO we should maintain at least one helper officially as a submodule to
ensure the git submodule mechanisms remain viable for distributors as a
modern replacement for the old squid-2 way of building their custom
helpers.
The above paragraph does not supply the information required for me to
answer your original question. To make progress, let's
Re: [squid-dev] RFC submodule repositories
Hey Amos, I believe we are trying to benefit something from the updates. CERN have been using my packages for a very long time now. But as you can see their updates are at: http://linuxsoft.cern.ch/mirror/ and it states latest update at 2019 so they have not upgraded their mirror for at-least 2-3 years now and I believe that they can stand by them self. I am merely packaging for who ever needs it. >From my point of view there are core helpers and deprecates helpers. For example ssl_crtd is one of the squid core helpers for couple groups for a very long time now. I can try to pack squid helpers as a stand alone package given that I will have something like what you suggest. We can try to make a list of the helpers and suggest a pass vote per each of them. Eliezer Eliezer Croitoru NgTech, Tech Support Mobile: +972-5-28704261 Email: ngtech1...@gmail.com Web: https://ngtech.co.il/ My-Tube: https://tube.ngtech.co.il/ -Original Message- From: squid-dev On Behalf Of Amos Jeffries Sent: Tuesday, 2 August 2022 5:32 To: squid-dev@lists.squid-cache.org Subject: Re: [squid-dev] RFC submodule repositories On 1/08/22 03:09, Alex Rousskov wrote: > On 7/31/22 00:29, Amos Jeffries wrote: >> When PR #937 merges we will have the ability to shuffle old helpers >> into a separate repository that users can import as a submodule to >> build with their Squid as-needed. > > In my experience, git submodules are a powerful feature that is rather > difficult to use correctly. Some of submodule aspects do not feel > "natural" to many humans. I am pretty sure that proper introduction of > submodules will require a lot of our time and nerve cells. What will we > optimize by introducing such a significant complication as git submodules? > ( So every change to Squid has to be justified as an optimization now. Right... ) We "optimize" future code maintenance workload with the ability to drop outdated helpers which are still required by a small group of users but no longer want to be actively developed by the core dev team. Case in point being CERN who still require our bundled LanManager helper(s) for some internal needs. That requires a lot of deprecated and rarely touched code being maintained for only one (albeit important) user case. That code could all be shuffled to a separate repository outside the official Squid release, but maintained by developers that support CERN needs. > >> What (if any) updates do we need to make to Anubis and other >> infrastructure so support git submodules ? > > That answer probably depends, in part, on what support guarantees we are > going to issue for those submodules and on what our submodule update > policy will be. Integrating two or more git repositories together is a > complicated issue... > IMO we should maintain at least one helper officially as a submodule to ensure the git submodule mechanisms remain viable for distributors as a modern replacement for the old squid-2 way of building their custom helpers. Cheers Amos ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
Re: [squid-dev] RFC submodule repositories
> > > ( So every change to Squid has to be justified as an optimization now. > Right... ) > My experience with submodules is not great; I often ended up with detached sub-branches that were a pain to deal with. Given that, I am trying to understand if it's worth it, that's all. As opposed to, say, looking for a way to installing the bits that are essential to building helpers in some sort of squid-dev package, so we can just spin off completely those helpers we don't want to maintain -- Francesco ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
Re: [squid-dev] RFC submodule repositories
As a package maintainer: the workflow for package builds on many OS uses tar downloads and if these are produced by git-archive (e.g. as done on github's "download source" links) they don't include submodules. So, if used for some helper which is normally included in OS packages, submodules can make it harder to test changes from git commits (for example, to see if a recently committed change fixes a problem). Of course a tar can be built locally if needed, just adds some extra steps, and perhaps there are benefits from submodules to outweigh this. ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
Re: [squid-dev] RFC submodule repositories
On 1/08/22 03:09, Alex Rousskov wrote: On 7/31/22 00:29, Amos Jeffries wrote: When PR #937 merges we will have the ability to shuffle old helpers into a separate repository that users can import as a submodule to build with their Squid as-needed. In my experience, git submodules are a powerful feature that is rather difficult to use correctly. Some of submodule aspects do not feel "natural" to many humans. I am pretty sure that proper introduction of submodules will require a lot of our time and nerve cells. What will we optimize by introducing such a significant complication as git submodules? ( So every change to Squid has to be justified as an optimization now. Right... ) We "optimize" future code maintenance workload with the ability to drop outdated helpers which are still required by a small group of users but no longer want to be actively developed by the core dev team. Case in point being CERN who still require our bundled LanManager helper(s) for some internal needs. That requires a lot of deprecated and rarely touched code being maintained for only one (albeit important) user case. That code could all be shuffled to a separate repository outside the official Squid release, but maintained by developers that support CERN needs. What (if any) updates do we need to make to Anubis and other infrastructure so support git submodules ? That answer probably depends, in part, on what support guarantees we are going to issue for those submodules and on what our submodule update policy will be. Integrating two or more git repositories together is a complicated issue... IMO we should maintain at least one helper officially as a submodule to ensure the git submodule mechanisms remain viable for distributors as a modern replacement for the old squid-2 way of building their custom helpers. Cheers Amos ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
Re: [squid-dev] RFC submodule repositories
Hey Amos, I have not seen too many updates to the helpers for a very long time. I think that separating the helpers will result in a specific build dependencies removal. If this is the case then we need to first consider what it will cause eventually. >From me as a builder I need a simple way to pull the full git or package. There are specific helpers which are based on squid sources so these might be affected by any chance. Correct me if I'm wrong about my assumption. To my opinion if the helpers are stable by themselves then they can be moved to another git repo. (Maybe for them to use the language default libraries is good enough) Not related directly but if you ask me some helpers should be deprecated since we are on 5.x+ and it means that their lifecycle ended. One example is the CentOS 6 usage to the favor of Debian 11 from Katerina. I need to learn more about git submodules but I prefer not to for now. Thanks, Eliezer Eliezer Croitoru NgTech, Tech Support Mobile: +972-5-28704261 Email: ngtech1...@gmail.com Web: https://ngtech.co.il/ My-Tube: https://tube.ngtech.co.il/ -Original Message- From: squid-dev On Behalf Of Amos Jeffries Sent: Sunday, 31 July 2022 7:29 To: Squid Developers Subject: [squid-dev] RFC submodule repositories When PR #937 merges we will have the ability to shuffle old helpers into a separate repository that users can import as a submodule to build with their Squid as-needed. What (if any) updates do we need to make to Anubis and other infrastructure so support git submodules ? Amos ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
Re: [squid-dev] RFC submodule repositories
Hi, I second Alex' view point, I don't have a clear understanding about what is the benefit that you're trying to obtain. On Sun, Jul 31, 2022 at 4:09 PM Alex Rousskov < rouss...@measurement-factory.com> wrote: > On 7/31/22 00:29, Amos Jeffries wrote: > > When PR #937 merges we will have the ability to shuffle old helpers into > > a separate repository that users can import as a submodule to build with > > their Squid as-needed. > > In my experience, git submodules are a powerful feature that is rather > difficult to use correctly. Some of submodule aspects do not feel > "natural" to many humans. I am pretty sure that proper introduction of > submodules will require a lot of our time and nerve cells. What will we > optimize by introducing such a significant complication as git submodules? > > > > What (if any) updates do we need to make to Anubis and other > > infrastructure so support git submodules ? > > That answer probably depends, in part, on what support guarantees we are > going to issue for those submodules and on what our submodule update > policy will be. Integrating two or more git repositories together is a > complicated issue... > > Alex. > ___ > squid-dev mailing list > squid-dev@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-dev > -- Francesco ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
Re: [squid-dev] RFC submodule repositories
On 7/31/22 00:29, Amos Jeffries wrote: When PR #937 merges we will have the ability to shuffle old helpers into a separate repository that users can import as a submodule to build with their Squid as-needed. In my experience, git submodules are a powerful feature that is rather difficult to use correctly. Some of submodule aspects do not feel "natural" to many humans. I am pretty sure that proper introduction of submodules will require a lot of our time and nerve cells. What will we optimize by introducing such a significant complication as git submodules? What (if any) updates do we need to make to Anubis and other infrastructure so support git submodules ? That answer probably depends, in part, on what support guarantees we are going to issue for those submodules and on what our submodule update policy will be. Integrating two or more git repositories together is a complicated issue... Alex. ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
