Re: [squid-dev] RFC: make FOLLOW_X_FORWARDED_FOR unconditional

[ngtech1ltd]

Hey,

Not sure I understood exactly what the proposal is?
>From Amos response I understand that it will be converted into some ACL which 
>can be configured or not.
Right?

Eliezer

From: squid-dev  On Behalf Of 
Francesco Chemolli
Sent: Tuesday, October 10, 2023 19:18
To: Squid Developers 
Subject: [squid-dev] RFC: make FOLLOW_X_FORWARDED_FOR unconditional

Hi all,
   what if we removed the configure option for FOLLOW_X_FORWARDED_FOR, and 
made it unconditionally part of Squid?

It is on by default, and it is controlled by runtime configuration, removing 
the compile-time option would ensure we have better testing coverage.

What do you think?

-- 
Francesco

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
https://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] RFC: GitHub Projects and Issues

[ngtech1ltd]

Hey Amos and Alex,

(Writing freely)

It's pretty simple to understand that there are two sides to the same picture 
so it's
possible that there are multiple scenarios.
To my understanding opening GitHub issues is not an explicit move that states 
any migration
from https://bugs.squid-cache.org/ towards any other option.
I want to believe that developers and sysadmins are smart enough to understand 
a declaration
on the GitHub issues template. (Assuming they will read the text..)

In general opening an issue is nice and will probably help to work with PR's 
but the main question is:
Will it bring more then there is today?

However, there is another issue.
The development team will need to be watch another vector in the development 
cycle and I believe
that it's already not a simple task as it is.

It's better to think about any way to address both the technical and the 
administrative overhead
that I might suspect it will leave the development team with before the full 
adoption.

I believe that for now restricting the issues with an experimental statement 
would be enough.
If Amos feels fine with handling all the issues related syn/syn-ack/ack of any 
issue being opened
by himself I believe it will not harm anyone.

HTH,
Eliezer

-Original Message-
From: squid-dev  On Behalf Of Alex 
Rousskov
Sent: Friday, May 5, 2023 6:51 PM
To: squid-dev@lists.squid-cache.org
Subject: Re: [squid-dev] RFC: GitHub Projects and Issues

On 5/5/23 09:39, Amos Jeffries wrote:

> You may (or not) have noticed that recently I have been experimenting 
> with GitHub Projects.
> Creating a few for the major long-term efforts and assigned a number of 
> the open PRs to them.
>
> IMO this looks like it could be a better way to track progress on 
> incomplete features or code conversions instead of wiki Feature pages.

I am against using GitHub Projects for projects that lack Squid Project 
consensus. Just like Feature pages, GitHub Projects currently create a 
false impression that the Squid Project has agreed that some activity is 
a good idea, that some details of that activity have been reviewed and 
approved by the Squid Project, and/or that some GitHub PRs match that 
good idea.

I wish you have not started using GitHub Projects before discussing that 
shared Squid Project resource use. Please pause that experiment.


> That limitation might be resolved by enabling GitHub Issues for (only) 
> feature-enhancement TODO items prior to PR creation.

Enabling GitHub Issues and then rejecting "regular" bug reports 
submitted via GitHub Issues is bad UX. IMO, we should either enable 
GitHub Issues for regular bug reports and other issues that folks expect 
to file via GitHub Issues (and deprecate Bugzilla) or not enable them at 
all.


HTH,

Alex.

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] RFC: Reject repeated same-name annotations

[ngtech1ltd]

Hey Alex,

I must admit that I didn't understand it enough to make sense on what specific 
scenario for example it will affect.
We have a set of sources for a "note" ie:
* "note" directive
* adaptation_meta directive
* annotate_transaction ACL [1]
* annotate_client ACL [1]
* adaptation services responses (eCAP and ICAP)
* helper responses

The only one I have used until today is the helpers and maybe once more with an 
ICAP service.

I couldn't make sense the 1+2 appended comments with the rejection RFC.
I assumed that what would happen is that if multiple helpers for example will 
use the same note or
a single response will contain multiple notes with the same key these will be 
appended.
The last time I have seen this I assumed it's in an array fashion ie multiple 
values compared to a single string.
I do not remember the subject enough but what I do remember is that there was 
an issue with
the comparison/check of a note ACL when two values were applied with the same 
name.

>From what I understood until now a single helper that will respond with 
>multiple note_1=v note_1=v
Will trigger a fatal error and I have mixed feelings about it.
However, if multiple helpers will send both each in it's turn a note_1=v these 
will be appended.

I agree that the result should be predictable however if logs can help to trace 
the issue I believe it's predicted enough
to not say about the current situation "un-predictable".
I would say that since it's not a "sync" engine which the timing belt must not 
miss a "beat" or a "tooth"
It's an async engine which is far more complex.

I hope I understood the RFC and what's above it so my words will make sense of 
themselves.

Eliezer


Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1...@gmail.com
Web: https://ngtech.co.il/
My-Tube: https://tube.ngtech.co.il/

-Original Message-
From: squid-dev  On Behalf Of Alex 
Rousskov
Sent: Thursday, 15 December 2022 23:30
To: Squid Developers 
Subject: [squid-dev] RFC: Reject repeated same-name annotations

Hello,

 I propose to adjust Squid code to reject repeated same-name 
annotations from each and every source that supplies annotations:

* "note" directive
* adaptation_meta directive
* annotate_transaction ACL [1]
* annotate_client ACL [1]
* adaptation services responses (eCAP and ICAP)
* helper responses

If this RFC is approved: A configuration that contains a directive with 
repeated same-name annotations will be rejected with a fatal ERROR[2]. A 
helper or service response that contains repeated same-name annotations 
will trigger a non-fatal (to Squid or transaction) cache.log ERROR[2].


Currently, Squid treats repeated same-name annotations inconsistently. 
Depending on the annotation source, Squid processing code may

* use the first same-name annotation and ignore repetitions
* use the last same-name annotation and ignore repetitions
* use all same-name annotations, honoring repetitions

These inconsistencies make it difficult to improve/enhance/optimize 
Squid code, while Squid ignorance hides misconfigurations and 
helper/service implementation bugs, including problems that may be 
related to access controls and other sensitive matters.


Any objections or better ideas?


Thank you,

Alex.

[1] In this context, we are talking about same-name annotations 
mentioned in the corresponding ACL _configuration_ (i.e. all "acl" 
directives with a given ACL name). A repeated _computation_ of 
annotate_foo ACL will continue to deal with same-name annotations as 
documented -- a "name+=value" configuration will continue to append 
values to the existing same-name annotation, while a "name=value" 
configuration will continue to overwrite any existing same-name annotation.

[2] Repeated same-name annotations that all have identical _values_ will 
be flagged with a WARNING instead. Some overly simplistic configuration 
generators, complicated configurations build from many include files, 
and dumb helpers/services might generate repeated same-everything 
annotations. Since such repetitions can be _safely_ ignored (honoring 
just one name=value pair among all the identical ones), we do not have 
to reject the configuration or log an ERROR because of them.
___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] RFC: Semaphore CI to GitHub Actions migration

[ngtech1ltd]

Sounds good.
 

Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1...@gmail.com  
Web: https://ngtech.co.il/
My-Tube: https://tube.ngtech.co.il/
 
From: squid-dev  On Behalf Of 
Francesco Chemolli
Sent: Thursday, 20 October 2022 12:41
To: Alex Rousskov 
Cc: Squid Developers 
Subject: Re: [squid-dev] RFC: Semaphore CI to GitHub Actions migration
 
Hi Alex,
   this all sounds sensible to me and in general a good idea
 
On Wed, Oct 19, 2022 at 3:25 PM Alex Rousskov mailto:rouss...@measurement-factory.com> > wrote:
Hello,

 I plan to gradually turn Semaphore CI testing off and make GitHub 
Actions required. We should not babysit the same tests in two setups. 
Here is the current status of CI tests with regard to Semaphore and 
GitHub Actions together with the corresponding planned actions:

1. Functionality tests: Essentially the same set of tests. Semaphore CI 
has one extra/old test but it is disabled for master/v6 code. The 
proxy-collapsed-forwarding test often fails on Semaphore, requiring a 
manual restart of all tests. The busy-restart test usually fails on 
GitHub Actions, but those failures are currently ignored.

Plan: I will leave the busy-restart test running on Semaphore CI until 
we find a way to make it stable in GitHub Actions environment. I will 
turn off the other Semaphore CI functionality tests and make the GitHub 
Actions ones required.


2. Source maintenance tests: The same set of tests. GitHub Actions have 
the right Astyle version, so the formatting test actually works on 
GitHub (but its results are currently ignored on both platforms).

Plan: I will turn off Semaphore CI source maintenance tests and make the 
GitHub Actions ones required instead. Formatting test results will still 
be ignored (that is a separate decision/change/action out of this thread 
scope; let's not discuss it here).


3. Build tests: Semaphore CI uses Ubuntu 14.04. GitHub Actions uses 
Ubuntu 22.04. Semaphore CI has fewer build dependencies installed. 
GitHub Actions do not provide Ubuntu 14.04 runners[1].

Plan: I will keep Semaphore CI build tests and make the GitHub Actions 
tests required. When Semaphore CI build tests start failing (e.g., 
because dependency repositories stop working "as is"), or when we stop 
supporting that old environment, I will disable those tests.


If you have any objections or better ideas about gradually moving away 
from Semaphore CI, please discuss.


Thank you,

Alex.

[1]  GitHub provides Ubuntu 18.04 runners, but they are deprecated, will 
purposefully _fail_ according to GitHub schedule, and will be removed in 
December. We should not use them. Details at 
https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22/
___
squid-dev mailing list
squid-dev@lists.squid-cache.org  
http://lists.squid-cache.org/listinfo/squid-dev


 
-- 
Francesco
___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

[squid-dev] security_file_certgen protocol

[ngtech1ltd]

Hey Everybody,

I am a bit lost in my search.
I am trying to write a service like security_file_certgen as a daemon that will 
be communicated  via a TCP or UNIX Socket.
However, it’s a bit hard for me now to grasp the STDIN/STDOUT protocol of 
security_file_certgen.
I remember vaguely that it involves reading from some string (else then new 
lines) to another and then sends back
to stdout a certificate string.

So what are the parts of the request object and what are the parts of the 
response object?
If I will grasp it I will be able to model it in a single ruby script.

I know this is not the first time I am asking about this and it’s harder for me 
that I forget such simple things.
I will be thankful for any help with this.

Eliezer


Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1...@gmail.com
Web: https://ngtech.co.il/
My-Tube: https://tube.ngtech.co.il/


___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] RFC submodule repositories

[ngtech1ltd]

Hey Amos,

I believe we are trying to benefit something from the updates.
CERN have been using my packages for a very long time now.
But as you can see their updates are at:
http://linuxsoft.cern.ch/mirror/

and it states latest update at 2019 so they have not upgraded their mirror for 
at-least 2-3 years now and I believe that they can stand by them self.
I am merely packaging for who ever needs it.
>From my point of view there are core helpers and deprecates helpers.
For example ssl_crtd is one of the squid core helpers for couple groups for a 
very long time now.

I can try to pack squid helpers as a stand alone package given that I will have 
something like what you suggest.

We can try to make a list of the helpers and suggest a pass vote per each of 
them.

Eliezer


Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1...@gmail.com
Web: https://ngtech.co.il/
My-Tube: https://tube.ngtech.co.il/

-Original Message-
From: squid-dev  On Behalf Of Amos 
Jeffries
Sent: Tuesday, 2 August 2022 5:32
To: squid-dev@lists.squid-cache.org
Subject: Re: [squid-dev] RFC submodule repositories

On 1/08/22 03:09, Alex Rousskov wrote:
> On 7/31/22 00:29, Amos Jeffries wrote:
>> When PR #937 merges we will have the ability to shuffle old helpers 
>> into a separate repository that users can import as a submodule to 
>> build with their Squid as-needed.
> 
> In my experience, git submodules are a powerful feature that is rather 
> difficult to use correctly. Some of submodule aspects do not feel 
> "natural" to many humans. I am pretty sure that proper introduction of 
> submodules will require a lot of our time and nerve cells. What will we 
> optimize by introducing such a significant complication as git submodules?
> 


( So every change to Squid has to be justified as an optimization now. 
Right... )

We "optimize" future code maintenance workload with the ability to drop 
outdated helpers which are still required by a small group of users but 
no longer want to be actively developed by the core dev team.

Case in point being CERN who still require our bundled LanManager 
helper(s) for some internal needs. That requires a lot of deprecated and 
rarely touched code being maintained for only one (albeit important) 
user case.
  That code could all be shuffled to a separate repository outside the 
official Squid release, but maintained by developers that support CERN 
needs.


> 
>> What (if any) updates do we need to make to Anubis and other 
>> infrastructure so support git submodules ?
> 
> That answer probably depends, in part, on what support guarantees we are 
> going to issue for those submodules and on what our submodule update 
> policy will be. Integrating two or more git repositories together is a 
> complicated issue...
> 

IMO we should maintain at least one helper officially as a submodule to 
ensure the git submodule mechanisms remain viable for distributors as a 
modern replacement for the old squid-2 way of building their custom helpers.


Cheers
Amos
___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] RFC submodule repositories

[ngtech1ltd]

Hey Amos,

I have not seen too many updates to the helpers for a very long time.
I think that separating the helpers will result in a specific build 
dependencies removal.
If this is the case then we need to first consider what it will cause 
eventually.
>From me as a builder I need a simple way to pull the full git or package.

There are specific helpers which are based on squid sources so these might be 
affected by any chance.

Correct me if I'm wrong about my assumption.

To my opinion if the helpers are stable by themselves then they can be moved to 
another git repo.
(Maybe for them to use the language default libraries is good enough)


Not related directly but if you ask me some helpers should be deprecated since 
we are on 5.x+
and it means that their lifecycle ended.

One example is the CentOS 6 usage to the favor of Debian 11 from Katerina.

I need to learn more about git submodules but I prefer not to for now.

Thanks,
Eliezer


Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1...@gmail.com
Web: https://ngtech.co.il/
My-Tube: https://tube.ngtech.co.il/

-Original Message-
From: squid-dev  On Behalf Of Amos 
Jeffries
Sent: Sunday, 31 July 2022 7:29
To: Squid Developers 
Subject: [squid-dev] RFC submodule repositories

When PR #937 merges we will have the ability to shuffle old helpers into 
a separate repository that users can import as a submodule to build with 
their Squid as-needed.


What (if any) updates do we need to make to Anubis and other 
infrastructure so support git submodules ?


Amos

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

[squid-dev] Errors while building 5.6 on Ubuntu 22.04

[ngtech1ltd]

Hey Everybody,
 
I have just started my build tests on Ubuntu 22.04  and I got some errors, 
something with OpenSSL 3.0.
I have seen that there has been a patch for 5.5 in Debian at:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1005650
 
 
but I have not see such a fix from the squid project.
Is there any usable patch I can try to apply to build and fix it?
 
The build errors:
### START
ibtool: compile:  g++ -DHAVE_CONFIG_H 
-DDEFAULT_CONFIG_FILE=\"/etc/squid/squid.conf\" 
-DDEFAULT_SQUID_DATA_DIR=\"/usr/share/squid\" 
-DDEFAULT_SQUID_CONFIG_DIR=\"/etc/squid\" -I/home/builder/squid-5.6 
-I/home/builder/squid-5.6/include -I/home/builder/squid-5.6/lib 
-I/home/builder/squid-5.6/src -I../../include -isystem /usr/include/mit-krb5 
-I/usr/include/libxml2 -Wall -Wpointer-arith -Wwrite-strings -Wcomments 
-Wshadow -Woverloaded-virtual -Werror -pipe -D_REENTRANT -I/usr/include/libxml2 
-m64 -I/usr/include/p11-kit-1 -g -O2 -MT old_api.lo -MD -MP -MF 
.deps/old_api.Tpo -c /home/builder/squid-5.6/src/mem/old_api.cc  -fPIC -DPIC -o 
.libs/old_api.o
In file included from /home/builder/squid-5.6/src/security/Session.h:14,
 from /home/builder/squid-5.6/src/security/forward.h:15,
 from /home/builder/squid-5.6/src/SquidConfig.h:26,
 from /home/builder/squid-5.6/src/mem/old_api.cc:24:
/home/builder/squid-5.6/src/security/forward.h: In function 'void 
Security::DH_free_cpp(DH*)':
/home/builder/squid-5.6/src/security/LockingPointer.h:34:21: error: 'void 
DH_free(DH*)' is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
   34 | function(a); \
/home/builder/squid-5.6/src/security/forward.h:96:1: note: in expansion of 
macro 'CtoCpp1'
   96 | CtoCpp1(DH_free, DH *);
  | ^~~
In file included from /home/builder/squid-5.6/compat/openssl.h:35,
 from /home/builder/squid-5.6/src/security/Context.h:15,
 from /home/builder/squid-5.6/src/security/forward.h:14,
 from /home/builder/squid-5.6/src/SquidConfig.h:26,
 from /home/builder/squid-5.6/src/mem/old_api.cc:24:
/usr/include/openssl/dh.h:200:28: note: declared here
  200 | OSSL_DEPRECATEDIN_3_0 void DH_free(DH *dh);
  |^~~
In file included from /home/builder/squid-5.6/src/SquidConfig.h:26,
 from /home/builder/squid-5.6/src/mem/old_api.cc:24:
/home/builder/squid-5.6/src/security/forward.h: At global scope:
/home/builder/squid-5.6/src/security/forward.h:97:70: error: 'int 
DH_up_ref(DH*)' is deprecated: Since OpenSSL 3.0 
[-Werror=deprecated-declarations]
   97 | typedef Security::LockingPointer > DhePointer;
  |  
^
In file included from /home/builder/squid-5.6/compat/openssl.h:35,
 from /home/builder/squid-5.6/src/security/Context.h:15,
 from /home/builder/squid-5.6/src/security/forward.h:14,
 from /home/builder/squid-5.6/src/SquidConfig.h:26,
 from /home/builder/squid-5.6/src/mem/old_api.cc:24:
/usr/include/openssl/dh.h:201:27: note: declared here
  201 | OSSL_DEPRECATEDIN_3_0 int DH_up_ref(DH *dh);
  |   ^
In file included from /home/builder/squid-5.6/src/SquidConfig.h:26,
 from /home/builder/squid-5.6/src/mem/old_api.cc:24:
/home/builder/squid-5.6/src/security/forward.h:97:79: error: 'int 
DH_up_ref(DH*)' is deprecated: Since OpenSSL 3.0 
[-Werror=deprecated-declarations]
   97 | typedef Security::LockingPointer > DhePointer;
  | 
  ^
In file included from /home/builder/squid-5.6/compat/openssl.h:35,
 from /home/builder/squid-5.6/src/security/Context.h:15,
 from /home/builder/squid-5.6/src/security/forward.h:14,
 from /home/builder/squid-5.6/src/SquidConfig.h:26,
 from /home/builder/squid-5.6/src/mem/old_api.cc:24:
/usr/include/openssl/dh.h:201:27: note: declared here
  201 | OSSL_DEPRECATEDIN_3_0 int DH_up_ref(DH *dh);
  |   ^
In file included from /home/builder/squid-5.6/src/ssl/support.h:21,
 from /home/builder/squid-5.6/src/SquidConfig.h:29,
 from /home/builder/squid-5.6/src/mem/old_api.cc:24:
/home/builder/squid-5.6/src/ssl/gadgets.h:61:51: error: 'void RSA_free(RSA*)' 
is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
   61 | typedef std::unique_ptr> 
RSA_Pointer;
  |   ^~~~
In file included from /usr/include/openssl/x509.h:36,
 from /usr/include/openssl/ssl.h:31,
 from /home/builder/squid-5.6/compat/openssl.h:44,
 from /home/builder/squid-5.6/src/security/Context.h:15,
 from /home/builder/squid-5.6/src/security/forward.h:14,
 

Re: [squid-dev] Squid + docker multi-distro test builds script

[ngtech1ltd]

Francesco,
 
This is a very good bash script.
 
Thanks!
 

Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1...@gmail.com  
Web: https://ngtech.co.il/
My-Tube: https://tube.ngtech.co.il/
 
From: squid-dev  On Behalf Of 
Francesco Chemolli
Sent: Sunday, 26 June 2022 14:03
To: Squid Developers 
Subject: [squid-dev] Squid + docker multi-distro test builds script
 
Hi all,
   in order to facilitate multiplatform (at least on Linux) development 
velocity, let me share a script I use for this purpose. It is based on the same 
tech we run our CI farm on.
 
the script is at 
https://github.com/kinkie/support-tools/blob/master/squid-build/do-build 
 
It's reasonably well documented via the standard --help mechanism. It's meant 
ot be run a Linux system with docker.
 
The main convenience argument is "--in " 
 
$ do-build --in debian-stable --all
 
will - as you may guess - run a bootstrap/configure/build/test cycle in the 
current working directory in a debian-stable runtime environment.
 
I hope this helps others increase their velocity on multi multi-distribution 
development. Please keep in mind I make no promises on this script - I 
occasionally tinker with it and it may break.
Feedback is welcome 
 
-- 
Francesco
___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev