Re: [squid-dev] [PATCH] External ACL helpers error handling & caching

2017-01-26 Thread Amos Jeffries
On 12/01/2017 8:21 a.m., Christos Tsantilas wrote:
> 
> The patch applied to squid-5 branch as r15005.
> I am also attaching the patch for squid-3.5
> 

Applied to v4 and 3.5.

Amos

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev


Re: [squid-dev] [PATCH] External ACL helpers error handling & caching

2017-01-11 Thread Amos Jeffries
On 10/01/2017 12:49 a.m., Christos Tsantilas wrote:
> The helper protocol for external ACLs [1] defines three possible return
> values:
>OK - Success. ACL test matches.
>ERR - Success. ACL test fails to match.
>BH - Failure. The helper encountered a problem.
> 
> The external acl helpers distributed with squid currently doesn't follow
> this definition. For example, upon connection error, ERR is returned:
> 
>$ ext_ldap_group_acl ... -d
>ext_ldap_group_acl: WARNING: could not bind to binddn 'Can't contact
> LDAP server'
>ERR
> 
>  This is does not allow to distinguish "no match" and "error" either and
> therefore negative caches "ERR", also in the case of an error.
> 
> Moreover there are multiple problems inside squid when trying to handle
> BH responses:
>   - Squid-5 and squid-4 retries requests for BH responses but crashes
> after the maximum retry number (currently 2) is reached.
>   - If an external acl helper return always BH (eg because the LDAP
> server is down) squid sends infinitely new request to the helper.
> 
> This patch fixes the problems described above.
> 
> This is a Measurement Factory project
> 

Thank you for this long overdue fix.

+1. Though if possible I would like one extra change...

Please add the below method to class external_acl and use it to
de-duplicate the complex if-conditions in external_acl_cache_touch and
external_acl_cache_add about whether a response is non-cacheable:

bool
external_acl::maybeCacheable(const allow_t )
{
if (cache_size <= 0)
return false; // cache is disabled

if (result == ACCESS_DUNNO)
return false; // non-cacheable response

if ((result == ACCESS_ALLOWED ? ttl : negative_ttl) <= 0)
return false; // not caching this type of response

return true;
}


Could you also mention for the squid-dev record how much testing this
patch has had.

Cheers
Amos

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev


[squid-dev] [PATCH] External ACL helpers error handling & caching

2017-01-09 Thread Christos Tsantilas
The helper protocol for external ACLs [1] defines three possible return 
values:

   OK - Success. ACL test matches.
   ERR - Success. ACL test fails to match.
   BH - Failure. The helper encountered a problem.

The external acl helpers distributed with squid currently doesn't follow 
this definition. For example, upon connection error, ERR is returned:


   $ ext_ldap_group_acl ... -d
   ext_ldap_group_acl: WARNING: could not bind to binddn 'Can't contact 
LDAP server'

   ERR

 This is does not allow to distinguish "no match" and "error" either 
and therefore negative caches "ERR", also in the case of an error.


Moreover there are multiple problems inside squid when trying to handle 
BH responses:
  - Squid-5 and squid-4 retries requests for BH responses but crashes 
after the maximum retry number (currently 2) is reached.
  - If an external acl helper return always BH (eg because the LDAP 
server is down) squid sends infinitely new request to the helper.


This patch fixes the problems described above.

This is a Measurement Factory project
External ACL helpers error handling & caching

The helper protocol for external ACLs [1] defines three possible return values:
   OK - Success. ACL test matches.
   ERR - Success. ACL test fails to match.
   BH - Failure. The helper encountered a problem.

The external acl helpers distributed with squid currently doesn't follow this
definition. For example, upon connection error, ERR is returned:

   $ ext_ldap_group_acl ... -d
   ext_ldap_group_acl: WARNING: could not bind to binddn 'Can't contact LDAP server'
   ERR

 This is does not allow to distinguish "no match" and "error" either and
therefore negative caches "ERR", also in the case of an error.

Moreover there are multiple problems inside squid when trying to handle BH
responses:
  - Squid-5 and squid-4 retries requests for BH responses but crashes after the
maximum retry number (currently 2) is reached.
  - If an external acl helper return always BH (eg because the LDAP server is
down) squid sends infinitely new request to the helper.

This is a Measurement Factory project

=== modified file 'src/acl/external/AD_group/ext_ad_group_acl.cc'
--- src/acl/external/AD_group/ext_ad_group_acl.cc	2017-01-01 00:12:22 +
+++ src/acl/external/AD_group/ext_ad_group_acl.cc	2017-01-09 10:02:17 +
@@ -802,58 +802,58 @@
 DefaultDomain = xstrdup(machinedomain);
 }
 debug("%s " VERSION " " SQUID_BUILD_INFO " starting up...\n", argv[0]);
 if (use_global)
 debug("Domain Global group mode enabled using '%s' as default domain.\n", DefaultDomain);
 if (use_case_insensitive_compare)
 debug("Warning: running in case insensitive mode !!!\n");
 
 atexit(CloseCOM);
 
 /* Main Loop */
 while (fgets(buf, HELPER_INPUT_BUFFER, stdin)) {
 if (NULL == strchr(buf, '\n')) {
 /* too large message received.. skip and deny */
 fprintf(stderr, "%s: ERROR: Too large: %s\n", argv[0], buf);
 while (fgets(buf, HELPER_INPUT_BUFFER, stdin)) {
 fprintf(stderr, "%s: ERROR: Too large..: %s\n", argv[0], buf);
 if (strchr(buf, '\n') != NULL)
 break;
 }
-SEND_ERR("Invalid Request. Too Long.");
+SEND_BH(HLP_MSG("Invalid Request. Too Long."));
 continue;
 }
 if ((p = strchr(buf, '\n')) != NULL)
 *p = '\0';  /* strip \n */
 if ((p = strchr(buf, '\r')) != NULL)
 *p = '\0';  /* strip \r */
 
 debug("Got '%s' from Squid (length: %d).\n", buf, strlen(buf));
 
 if (buf[0] == '\0') {
-SEND_ERR("Invalid Request. No Input.");
+SEND_BH(HLP_MSG("Invalid Request. No Input."));
 continue;
 }
 username = strtok(buf, " ");
 for (n = 0; (group = strtok(NULL, " ")) != NULL; ++n) {
 rfc1738_unescape(group);
 groups[n] = group;
 }
 groups[n] = NULL;
 numberofgroups = n;
 
 if (NULL == username) {
-SEND_ERR("Invalid Request. No Username.");
+SEND_BH(HLP_MSG("Invalid Request. No Username."));
 continue;
 }
 rfc1738_unescape(username);
 
 if ((use_global ? Valid_Global_Groups(username, groups) : Valid_Local_Groups(username, groups))) {
 SEND_OK("");
 } else {
 SEND_ERR("");
 }
 err = 0;
 }
 return 0;
 }
 

=== modified file 'src/acl/external/LDAP_group/ext_ldap_group_acl.cc'
--- src/acl/external/LDAP_group/ext_ldap_group_acl.cc	2017-01-01 00:12:22 +
+++ src/acl/external/LDAP_group/ext_ldap_group_acl.cc	2017-01-09 10:44:19 +
@@ -454,166 +454,175 @@
 HMODULE WLDAP32Handle;
 
 WLDAP32Handle = GetModuleHandle("wldap32");
 if ((Win32_ldap_start_tls_s = (PFldap_start_tls_s) GetProcAddress(WLDAP32Handle, LDAP_START_TLS_S)) == NULL) {