Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-24 Thread Alex Rousskov 
On 07/24/2015 05:26 AM, Amos Jeffries wrote:
 I think you still misunderstand the OppSec RFC meanings.

Ditto.


 Since SSL support first went into Squid back in 1998 we have allowed
 cache_peer to connect to a remote https_port and sent http:// traffic
 over it.

Using the above as a sub-case of the use case we are considering, a
third independent proxy intercepting that peer-to-peer link (among other
traffic) should send bumped http:// messages from that link to a remote
ICAP service using Secure ICAP. To configure that desirable behavior,
the intercepting proxy admin may use received_encrypted.

In short, what is optional or opportunistic for some is often
required or highly desirable for others. Independent intercepting
intermediaries should honor others' decision to use encryption, even
when that use was optional or opportunistic. There are many
exceptions and wrinkles to these simple rules of thumb, and the admin
has the power to configure them.


 * the received part of the AC type name seems a bit odd.
  - perhapse connections_encrypted makes more sense with the
 description for it, and focus on TLS connections.

I disagree that connections_encrypted is better than received_encrypted,
especially in the presence of connectionless eCAP and
half-connectionless hits. A more general name may make this ACL more
forward-compatible.

However, if you insist, we will use your connections_encrypted name to
avoid discussing this further [unless somebody else suggests a better
name that you agree to, of course].


 * you mentionend cache HITs earlier.
  - IMO you can drop all mention of cache HIT in the commit message on
 grounds that no reply connection was involved with HIT at the scope
 level of current transaction.

The new ACL is evaluated against the master transaction (a set of
messages), not reply connection, so we have to document what happens to
cache hits IMO. I hope that you will not object to more explicit
documentation as long as it is correct, matches the code, etc.


 in src/HttpMsg.h:
 
 * Unless you are going to add secure eCAP flag functionality as part of
 ths patch scope I dont see why we should have the extra code or enum for
 srcEcaps existing yet.
  - please remove srcEcaps, or implement the ecaps flag feature.
  - TODO in places where the ecaps flag feature would need to hook in
 seems sufficient to me at present.
  - what remains is plain eCAP which does not naturally involve network
 connections, TLS or otherwise.
  - marking eCAP involvement does not seem useful either until that ecaps
 feature is added. Please consider turning those into TODO as well,
 though I will accept the current code for srcEcap.

I disagree that a TODO comment is better than correct, small, and
symmetric code, but we will replace eCAP-related code with TODO comments
to avoid prolonging this discussion.


 * can you explain the rational of doing this with a bitmask instead of a
 default-true 'tlsEncrypted' boolean flag in class HttpMsg ?

IIRC, there were two reasons for using distinct, named flags:

1. They simplify adding received_encrypted parameters that adjust how
the ACL is applied/computed.

2. tlsEncrypted or similar does not work well for connection-less
things like eCAP and hits. By using protocol/module names, we were able
to avoid this naming problem.

Distinct flags also serve as an excellent way of finding various message
sources in Squid source code, but we probably did not realize that when
designing the code.

If you insist, we can use boolean storage OR reduce the storage member
size to 16 bits (let us know what would you prefer).


Your other comments not discussed above should be addressed in the next
patch version. Please comment on what changes discussed above you insist on.


Thank you,

Alex.

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-24 Thread Amos Jeffries 
On 25/07/2015 3:28 a.m., Alex Rousskov wrote:
 On 07/24/2015 05:26 AM, Amos Jeffries wrote:
 I think you still misunderstand the OppSec RFC meanings.
 
 Ditto.
 
 
 Since SSL support first went into Squid back in 1998 we have allowed
 cache_peer to connect to a remote https_port and sent http:// traffic
 over it.
 
 Using the above as a sub-case of the use case we are considering, a
 third independent proxy intercepting that peer-to-peer link (among other
 traffic) should send bumped http:// messages from that link to a remote
 ICAP service using Secure ICAP. To configure that desirable behavior,
 the intercepting proxy admin may use received_encrypted.
 
 In short, what is optional or opportunistic for some is often
 required or highly desirable for others. Independent intercepting
 intermediaries should honor others' decision to use encryption, even
 when that use was optional or opportunistic. There are many
 exceptions and wrinkles to these simple rules of thumb, and the admin
 has the power to configure them.
 

Good point. I can agree with that.

 
 * the received part of the AC type name seems a bit odd.
  - perhapse connections_encrypted makes more sense with the
 description for it, and focus on TLS connections.
 
 I disagree that connections_encrypted is better than received_encrypted,
 especially in the presence of connectionless eCAP and
 half-connectionless hits. A more general name may make this ACL more
 forward-compatible.
 
 However, if you insist, we will use your connections_encrypted name to
 avoid discussing this further [unless somebody else suggests a better
 name that you agree to, of course].

I think insist. With that [] caveat.

Regards the ecaps relationship. Its already been mentiond that ecap is
secure by default being just a library attachment. AFAIK that normally
a local machine memory-only thing. What would make eCAP services worthy
of the in-secure tag would usually be fetching/sending the data
externally over some type of unsecured connections. Just like ICAP vs
ICAPS. Which brings in the connection meaning to eCAP as well in an
only slightly twisted way.

Regards the half-connectionless hits. If the prior transactions
involved with the cache data matter that much they should be marked as
secure for https:// and insecure for http://.
 BUT, you convinced me earlier that the scope of the ACL was deciding if
the current transactions connections all used TLS (or not). Which means
that pulling details about past transactions into the current one is a
bit out of scope.
 The half-connection they do have is already marked in the sources data.
So that fits connections as well if one squints sideways.


 
 * you mentionend cache HITs earlier.
  - IMO you can drop all mention of cache HIT in the commit message on
 grounds that no reply connection was involved with HIT at the scope
 level of current transaction.
 
 The new ACL is evaluated against the master transaction (a set of
 messages), not reply connection, so we have to document what happens to
 cache hits IMO. I hope that you will not object to more explicit
 documentation as long as it is correct, matches the code, etc.
 

No I dont think I object to correct documentation :-)


What I meant was that the code was not actually doing any reply markings
in regards to cache HITs that I could see. So it seems irrelevant to
mention their lack of interaction with a non-existent reply connection
when it is not affecting the outcome.


 
 in src/HttpMsg.h:

 * Unless you are going to add secure eCAP flag functionality as part of
 ths patch scope I dont see why we should have the extra code or enum for
 srcEcaps existing yet.
  - please remove srcEcaps, or implement the ecaps flag feature.
  - TODO in places where the ecaps flag feature would need to hook in
 seems sufficient to me at present.
  - what remains is plain eCAP which does not naturally involve network
 connections, TLS or otherwise.
  - marking eCAP involvement does not seem useful either until that ecaps
 feature is added. Please consider turning those into TODO as well,
 though I will accept the current code for srcEcap.
 
 I disagree that a TODO comment is better than correct, small, and
 symmetric code, but we will replace eCAP-related code with TODO comments
 to avoid prolonging this discussion.
 

Thank you.

I just realized after writing the above about eCAP that with a loaded
library being effectively secure by default and only certain uncommon
external activities making it non-secure. Then marking the traffic as
srcEcap == Unsafe before the ecaps flag exists is decreasing the amount
of visibly secure transactions. Erring on the side of insecurity rather
than security. Which is much of a sameness, but less desirable in common
cases.


 
 * can you explain the rational of doing this with a bitmask instead of a
 default-true 'tlsEncrypted' boolean flag in class HttpMsg ?
 
 IIRC, there were two reasons for using distinct, named flags:
 
 1. They simplify adding

Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-23 Thread Amos Jeffries 
On 23/07/2015 3:32 a.m., Alex Rousskov wrote:
 On 07/21/2015 04:25 AM, Amos Jeffries wrote:
 On 21/07/2015 9:42 a.m., Alex Rousskov wrote:
 adaptation_access icapS aclIcap
 adaptation_access icapN !aclIcap


 aclIcap can be a received_encrypted ACL. What ACL expression would you
 suggest for aclIcap if received_encrypted is not available?
 
 
   # top 5 criteria - TLS transactions
   acl aclIcap allof HTTPS
 
 What is HTTPS ACL? If it is based on the URI scheme, then AFAIK, it is
 not going to work because it will not match bumped http:// requests (at
 least).
 
 Good you have seen those. That is what the HTTPbis WG call
 opportunistic encryption traffic.
 
 
 I bet many (most?) of them are actually regular HTTP requests sent over
 TLS/SSL connections. There is nothing opportunistic about them.

That is the very definition of opportunistic in the related HTTP
documentation.

[Sorry wording gets flicked from security to encryption in those WG
threads a lot, didn't mean to bring it over here too].

RFC 7435 section 1.2 - A New Perspective

   Opportunistic Security (OS) is defined as the use of cleartext as
   the baseline communication security policy, with encryption and
   authentication negotiated and applied to the communication when
   available.


[Note that *when available* wording.]

http://tools.ietf.org/html/draft-ietf-httpbis-http2-encryption-02

   Serving https URIs require acquiring and configuring a valid
   certificate, which means that some deployments find supporting TLS
   difficult.  This document describes a usage model whereby sites can
   serve http URIs over TLS without being required to support strong
   server authentication.

   Opportunistic Security [RFC7435] does not provide the same guarantees
   as using TLS with https URIs; it is vulnerable to active attacks,
   and does not change the security context of the connection.
   Normally, users will not be able to tell that it is in use (i.e.,
   there will be no lock icon).

[Note that *serve http URIs over TLS without being required* and *does
not change the security context of the connection* wording.]

Both HTTP and HTTPS are regular message syntax. Both MAY transit over
TLS. The *only* difference is the URL scheme name indicating whether TLS
layer security is mandatory or optional/opportunistic.

 
 This is where the security requirements and your ACL use-case differ.
 Security requirements for those requests is that they be handled as per
 any other http:// - any encrytpion or security applied is *optional*.
 
 No requirement can _force_ something to be optional! If the use case

I think you misunderstand me.

The forcing applied is: optional-mandatory.

It is done by using the received_encrypted ACL. Apparently without admin
intending to.


 demands that requests received over TLS are sent to icapS service, then
 they should be sent to the icapS service, even if some of the
 corresponding request owners would not mind to see their requests going
 to the icapN service.

If you recall my very first words on this thread. I question the use-case.

 I object to making this ACL available *for this use case*.
[new emphasis].


Look closely:

+if (!(filled-request-sources  HttpMsg::srcUnsafe)  ||
+(filled-reply  !(filled-reply-sources  HttpMsg::srcUnsafe)))
+return 1;
+
+return 0;

The permutations of match/non-match vs traffic source vs mask setters is
counter intuitive.

eg. when policy wants traffic only from TLS sources to use icapS the
rest to use icapN.

aka.
  adaptation_access icapS allow aclReceivedEncrypted
  adaptation_access icapN allow !aclReceivedEncrypted

aka.
 icapN to get used on the inverted union of whether both request and
reply are not unsafe.

aka.
   if (!0 || !0) ... icapS HTTPS proper handling (okay)
   if (!1 || !0) ... icapS opportunistic security (oops)
   if (!0 || !1) ... icapS opportunistic security (oops)
   if (!1 || !1) ... icapN HTTP normal handling (okay)


While I personally like the outcome (more security). Its clearly not
matching the use case presented as *sole* reason for the ACLs existence.
Nor the apparent admin policy in adaptation_access.


So I ask; why are we bothering with this ACL?
 Just use an https:// scheme match, which will only be true in the top
line of above if()-matrix. Admin *wanting* opportunistic security
should not be the ones trying to avoid using icapS in the first place.


[I think that sums up what we are debating over. So I will refrain from
replying to the other sidetrails we got into.]

Amos
___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-23 Thread Alex Rousskov 
On 07/23/2015 01:41 PM, Tsantilas Christos wrote:
 On 07/23/2015 07:21 PM, Alex Rousskov wrote:
 Furthermore, the values of unsafe srcX enum constants should be
 increased to actually match the srcUnsafe mask (16 is still smaller than
 0x).

 This is should be OK.
 The safe flags are from:
  (1  (16 + 0)) = (1  (16 + 0))  0x
 to:
   (1  (16 + 15)) = (1  (16 + 15))  0x

You are right. I missed the shift operator!

Alex.

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-23 Thread Alex Rousskov 
On 07/23/2015 07:41 AM, Amos Jeffries wrote:
 On 23/07/2015 3:32 a.m., Alex Rousskov wrote:
 On 07/21/2015 04:25 AM, Amos Jeffries wrote:
 On 21/07/2015 9:42 a.m., Alex Rousskov wrote:
 adaptation_access icapS aclIcap
 adaptation_access icapN !aclIcap


 aclIcap can be a received_encrypted ACL. What ACL expression would you
 suggest for aclIcap if received_encrypted is not available?


   # top 5 criteria - TLS transactions
   acl aclIcap allof HTTPS

 What is HTTPS ACL? If it is based on the URI scheme, then AFAIK, it is
 not going to work because it will not match bumped http:// requests (at
 least).

 Good you have seen those. That is what the HTTPbis WG call
 opportunistic encryption traffic.


 I bet many (most?) of them are actually regular HTTP requests sent over
 TLS/SSL connections. There is nothing opportunistic about them.
 
 That is the very definition of opportunistic in the related HTTP
 documentation.


Hi Amos,

The opportunistic security (OS) documents you site talk about
optionally applying encryption to traffic X, to arrive at traffic XE. A
general-purpose intermediary cannot use those documents to go backwards
and say that any applied encryption was optional if we see traffic XE!
This backward direction is possible only when X or XE includes some sort
of an it is OK to decrypt at any time flag. Until such a flag is
defined for HTTP and/or TLS, and Squid learns to interpret it, the
discussion about opportunistic encryption is pretty much not relevant to
the received_encrypted ACL.

Moreover, the received_encrypted ACL itself and the related use cases do
not become wrong or obsolete when/if it is OK to decrypt at any time
flag is supported.

Is http://; scheme such a flag? Not for Squid. That URI scheme existed
long before any of the HTTP WG proposals you site were written, so Squid
should not assume that an encrypted http:// request is the result of OS.
Either a more reliable flag or an admin decision is required to enable
such behavior. All of this is orthogonal to received_encrypted.



 This is where the security requirements and your ACL use-case differ.
 Security requirements for those requests is that they be handled as per
 any other http:// - any encrytpion or security applied is *optional*.

 No requirement can _force_ something to be optional! If the use case
 
 I think you misunderstand me.
 
 The forcing applied is: optional-mandatory.
 
 It is done by using the received_encrypted ACL. Apparently without admin
 intending to.

Any assertion that uses a received ACL forces ... pattern is wrong. An
ACL is a yes/no test, nothing more. Yet, you keep blaming the ACL for
doing something other than testing. That does not compute for me.


 demands that requests received over TLS are sent to icapS service, then
 they should be sent to the icapS service, even if some of the
 corresponding request owners would not mind to see their requests going
 to the icapN service.
 
 If you recall my very first words on this thread. I question the use-case.

What does that mean? Are you saying the use case does not exist? That I
made it up? Or that we should not allow the admins to do what they want
to do? Something else?


 Look closely:

 +if (!(filled-request-sources  HttpMsg::srcUnsafe)  ||
 +(filled-reply  !(filled-reply-sources  HttpMsg::srcUnsafe)))
 +return 1;
 +
 +return 0;

That code does not make sense to me. I think it should be written like
this instead:

const bool safeRequest =
!(filled-request-sources  HttpMsg::srcUnsafe);
const bool safeReply = !filled-reply ||
!(filled-reply-sources  HttpMsg::srcUnsafe);
return (safeRequest  safeReply) ? 1 : 0;

Furthermore, the values of unsafe srcX enum constants should be
increased to actually match the srcUnsafe mask (16 is still smaller than
0x).


 While I personally like the outcome (more security). Its clearly not
 matching the use case presented as *sole* reason for the ACLs existence.
 Nor the apparent admin policy in adaptation_access.

To me, it just looks like an implementation bug. Christos, do you think
the code should be rewritten as in my sketch above?


 So I ask; why are we bothering with this ACL?
  Just use an https:// scheme match, which will only be true in the top
 line of above if()-matrix. Admin *wanting* opportunistic security
 should not be the ones trying to avoid using icapS in the first place.

I have answered that question several times already, including providing
specific examples requested by Kinkie: The URI scheme alone does not
match what the ACL is defined to match.


Cheers,

Alex.

___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-21 Thread Tsantilas Christos 

On 07/21/2015 01:25 PM, Amos Jeffries wrote:


No. Christos wrote this:

NOTE: Currently there is not any mechanism to indicate if a cached
object came from secure source or not, so we assume that all hits for
secure requests are secure too.


The cache hits rely on the request markings to determine the HIT
matching. In this case we have a https:// (secure, TLS-received) marked
request being re-written with non-TLS URL and delivered a HIT originated
from a non-TLS server.



If the HITs is the problem, it is something can be solved. Just in the 
application with the ICAP services described it is not needed.

Handling the HITs can be  marked as a TODO on this patch.


___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-21 Thread Amos Jeffries 
On 21/07/2015 9:42 a.m., Alex Rousskov wrote:
 On 07/20/2015 01:45 PM, Amos Jeffries wrote:
 On 21/07/2015 6:48 a.m., Alex Rousskov wrote:
 On 07/20/2015 09:27 AM, Kinkie wrote:
 So in my opinion the easiest way to move the discussion forward is to:

 1. find one use-case which cannot be covered by existing features

 Absolute impossibility is too high of a bar, IMO. A lot of things are
 possible with excessive amount of work and mind boggling complexity.
 However, I can suggest this configuration sketch as one example where
 the proposed ACL would be very handy if not essential:

 # A set of ports that may be changed depending on deployment.
 # Some may be configured to bump traffic, some not. Some may intercept.
 # The bumping ports do not bump everything (see ssl_bump below).
 http_port port1 ssl-bump ...
 https_port port2 ssl-bump ...
 http_port port3 ...
 https_port port4 ssl-bump ...

 # SslBumping rules (note that some of these ACLs cannot be computed
 # until later bumping stages, some rules create fake CONNECT requests
 # for adaptation services, and the order of rules is significant):
 ssl_bump splice aclBump0
 ssl_bump peek aclBump1
 ssl_bump stare aclBump2
 ssl_bump splice aclBump3
 ssl_bump bump aclBump4

 # If all transaction messages were received over SSL or TLS
 # connections, then send it to the ICAP service icapS. Otherwise,
 # send it to icapN. The rules apply to all transactions, including
 #  - plain http: transactions over plain connections to http_port,
 #  - plain http: transactions over intercepted plain connections,
 #  + plain http: transactions from bumped TLS connections,
 #  + plain http: transactions from direct TLS connections to https_port,
 #  + plain http: transactions from direct TLS connections to https_port
 #  + fake CONNECT messages generated for bumped intercepted transactions
 
 
 The fake connect request was not received over TLS.
 Conceptually it was a plain-TCP SYN received prior to TLS. No different
 in meaning to the non-fake CONNECT requests.
 
 This is debatable for fake CONNECT requests that are based on SSL/TLS
 information (SNI and such), but probably not worth discussing right now
 because it is a minor issue. We should document how [different kinds of]
 fake CONNECTs are classified.
 
 
 adaptation_access icapS aclIcap
 adaptation_access icapN !aclIcap


 aclIcap can be a received_encrypted ACL. What ACL expression would you
 suggest for aclIcap if received_encrypted is not available?


   # top 5 criteria - TLS transactions
   acl aclIcap allof HTTPS
 
 What is HTTPS ACL? If it is based on the URI scheme, then AFAIK, it is
 not going to work because it will not match bumped http:// requests (at
 least).

Good you have seen those. That is what the HTTPbis WG call
opportunistic encryption traffic.

This is where the security requirements and your ACL use-case differ.
Security requirements for those requests is that they be handled as per
any other http:// - any encrytpion or security applied is *optional*.
recieved_encrypted ACL seems to assume they are broken HTTPS and matches
for them.

What I'm getting at here is that these are more good candidates for the
performance-oriented admin to skip ICAPS processing if they want. Even
though they were bumped. The new ACL fails to let that happen (in a good
way, but still).


 
   # last 1 - the fake-CONNECTs but not the regular CONNECTs:
   acl bumpPorts myportname port2 port4
   acl aclIcap allof CONNECT !bumpPorts
 
 I am not sure this is correct because there may be CONNECT requests
 *inside* bumped SSL/TLS streams on port1 (at least):
 
  1. Receive CONNECT at http_port.
  2. Bump.
  3. Parse the first bumped HTTP request on the SSL/TLS channel.
  4. It is a CONNECT request!
 
 Sorry, I do not remember what Squid does with these nested CONNECTs
 right now, but they do exist in the wild.
 
 
 False-negative:

 Imagine that icapN was REQMOD and icapS a RESPMOD service. With
 URL-rewrite, eCAP or such modification of the reply making it https://
 (tainted).
 
 I assume you meant making it http:// (tainted).
 

No, this example eCAP taints it but leaves it as https://. So both the
inbound and outbound connections are fully using TLS. But the ACL
matches as if it were non-TLS connections.
 This will conflict with later cached handling, and logging info. Which
properly treats is as fully TLS received.


 
 Using received_encrypted the transactions;
  during request will match, so icapN does not get used.
  during reply will not match, so icapS does not get used.
 
 Kinkie has not asked for false-negatives so you are moving the goal
 posts. More importantly, I do not see a false negative in your example.
 Yes, an eCAP adaptation will taint the transaction, but it does not
 lead to a false negative, just a true negative (i.e., the negative
 decision matches what the admin should expect in this case).

They expect that two fetches of the same URL from the same client one
MISS gets marked non-TLS-received the

Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-20 Thread Kinkie 
Hi,
  sorry for butting in but I am a bit confused by this discussion, as it
seems to be straying from the technical merit; this is my attempt at
getting back to the core of the topic.

Amos claims that its stated objective can be achieved by other,
already-existing, features, and that it this proposal has a high
false-positive rate.

So in my opinion the easiest way to move the discussion forward is to:
1. find one use-case which cannot be covered by existing features
2a. claim that the false-positive scenarios found by Amos are either
intentional or not incorrect, OR
2b. find ways to reduce the false positives in the scenarios hihglighted by
Amos, OR
2c. rework the existing features (if possible) to allow the early
detection which is in my understanding the most touted benefit of the
proposed solution

Am I missing something obvious?

  Kinkie
___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev

Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-19 Thread Amos Jeffries 
On 18/07/2015 7:08 a.m., Alex Rousskov wrote:
 On 07/17/2015 11:48 AM, Amos Jeffries wrote:
 On 18/07/2015 3:13 a.m., Tsantilas Christos wrote:
 This patch adds received_encrypted ACL

 The new received_encrypted ACL matches transactions where all HTTP
 messages were received over TLS or SSL transport connections, including
 messages received from ICAP servers.
 
 Use case: Sending everything to Secure ICAP services increases
 adaptation performance overhead. Folks want to send received_encrypted
 transactions and only those transactions to Secure ICAP services.
 
 
 -1 on principle I object to making this ACL available for this use case.
 
 
 I must have read this wrong. You are not trying to block a useful
 feature simply because you dislike a specific use case, are you?

I am debating whether this is a useful feature. The one (and only) use
case presented for its existence is flawed.


 
 Given your comments below, it sounds like you misunderstood the use case
 itself, but even if you were right about that use case, blocking a
 feature based on a single use case seems a bit too harsh!
 
 
 And the (very) long answer:

 This is conceptually very bad.
 
 What exactly is this and why is it very bad?

what: The use-case behaviour being proposed as desirable.
why: for the reasons the rest of my mail was attempting to describe.

 
 * https:// scheme traffic is *supposed* to be kept secure no matter
 what. Either they need to obey that requirement or they dont (due to
 non-TLS security).

 * http:// and the other non-secure schemes are optional.
 
 Please do instead encourage using the request URI scheme (proto ACL)
 to make that determination of whether ICAP security is desirable or not.
 
 Looking at the request URI scheme does not work well: For example, the
 URI scheme may be insecure http://; but the message has been received
 via a secure channel (e.g., https_port or SslBump).

Exactly my point. ICAPS usage on these requests is *optional*. Any use
of TLS on the inbound/outbound connections is opportunistic.

If the admin want to reduce costs they are free to send it through
icap:// services with no unexpected danger. Use of received_encrypted
would unnecessarily *raise* their ICAPS load.


 
 
 In other news; HTTP is growing this thing called opportunistc security
 (aka. encryption). Where http:// schemed requests will be arriving over
 TLS, and maybe even going out via it and it traverses fully secure
 components. HTTPbis WG are already finding problems with servers not
 handling the schemes right in these conditions. Lets not add another
 bunch of bad assumptions to the mix. Follow the scheme.
 
 That seems like a counter-example to your own argument! The
 received_encrypted ACL should match if an http:// request arrived over
 TLS, and Squid may be able to offer better security for that
 transaction (e.g., by sending it to a Secure ICAP service or bypassing
 ICAP completely).

Only if you misunderstand the security requirements around
oppportunistic vs HTTPS.

Opportunistic security (http:// over TLS) means if the admin is *able*
and *willing* to pay the extra overhead costs of TLS, great. Otherwise
there is no pressure to do so.
 - in these cases the proposed patch adds nothing.

HTTPS (https://) means the admin is expected (almost mandatory) to pay
those costs. Failing to do so is a vulnerability.
 - in these cases the proposed patch adds only the ability to turn a
small vulnerability, into a bigger one. Existing ACLs already allows that.
 -if we want to be strict about hard-coding these requirements inside
Squid great. But then no need for the ACL config options.

So far its looking like there is no need for the ACL ... but lets go deeper.

 
 Also;
 1) we know about the directly connected security. But the connections
 beyond that may be insecure. You note this yourself about eCAP. Same
 goes for TLS connections!!
 
 Not sure what you are implying by that. Yes, the admin only knows about
 things under his or her control. How is that related to the proposed
 ACL? In the ICAP context, the proposed ACL simply taints transactions
 received from un-encrypted sources.

NP: The proposed ACL implies that Squid is able to determine security
levels of different connections. (safe/unsafe, tainted/untained).

All three of these listed situations are cases where that implication
fails in subtle ways that can bite badly by *raising* the unnecessary
ICAPS usage for admin who want to avoid it.

 
 We could go further and add a this ICAP service taints messages flag
 to [Secure] ICAP service configuration but there has not been any
 requests for that feature so perhaps adding more flags would be premature.
 
 
 2) Even when something is supposedly secure it can have wildly differing
 degrees of security from other things. TLS itself has NULL-cipher with
 zero security.
 
 How is that related to the proposed ACL? Are you implying that
 received_encrypted is the wrong name because NULL-ciphers exist? I
 hope you

Re: [squid-dev] [PATCH] received_encrypted ACL

2015-07-19 Thread Alex Rousskov 
On 07/19/2015 05:35 AM, Amos Jeffries wrote:
 On 18/07/2015 7:08 a.m., Alex Rousskov wrote:
 On 07/17/2015 11:48 AM, Amos Jeffries wrote:
 On 18/07/2015 3:13 a.m., Tsantilas Christos wrote:
 This patch adds received_encrypted ACL

 The new received_encrypted ACL matches transactions where all HTTP
 messages were received over TLS or SSL transport connections, including
 messages received from ICAP servers.

 Use case: Sending everything to Secure ICAP services increases
 adaptation performance overhead. Folks want to send received_encrypted
 transactions and only those transactions to Secure ICAP services.


 -1 on principle I object to making this ACL available for this use case.


 I must have read this wrong. You are not trying to block a useful
 feature simply because you dislike a specific use case, are you?


 I am debating whether this is a useful feature.


No, you are not. If you wanted to debate the usefulness of a feature,
then you would have not voted at all or voted -0 or above. Your -1 vote
is not a debate, it is a [blocking] resolution a.k.a. a veto.


 This is conceptually very bad.

 What exactly is this and why is it very bad?
 
 what: The use-case behaviour being proposed as desirable.

The desirability of that behavior is a fact -- Squid admins desire it.
We rarely spend a few days implementing something theoretically
desirable. The proposed feature satisfies real desires of real Squid admins.


 why: for the reasons the rest of my mail was attempting to describe.

I do not think the rest of your email explains why sending received
encrypted traffic to Secure ICAP services is conceptually very bad.
You talked about

* the need to keep https:// traffic secure
  (no one argues against that!)

* identifying secure traffic using ports and URI schemes
  (ports/schemes do not work well enough or we would not add this ACL)

* Squid's knowledge being limited to direct connections
  (true but admins often know more and can use ACLs accordingly)


 Looking at the request URI scheme does not work well: For example, the
 URI scheme may be insecure http://; but the message has been received
 via a secure channel (e.g., https_port or SslBump).

 Exactly my point. ICAPS usage on these requests is *optional*. Any use
 of TLS on the inbound/outbound connections is opportunistic.

The admin needs to send messages received via a SSL/TLS channel to a
Secure ICAP service. This routing is not optional. This routing is
required! There is nothing opportunistic or optional here.

You may argue that the admin's requirement itself is wrong, but you are
unlikely to win that argument. It may not be perfect, but it is very
reasonable in many environments.


 If the admin want to reduce costs they are free to send it through
 icap:// services with no unexpected danger. Use of received_encrypted
 would unnecessarily *raise* their ICAPS load.

Perhaps you have some other use case in mind, but in the use case at
hand, the admin cannot send received-encrypted traffic to just any
service. The external requirements outside our control or knowledge
require the admin to route received-encrypted traffic to special ICAPS
services.

It is easy (IMO) to think of many other use cases for the same feature:

* Use a special adaptation service for received-encrypted traffic.
* Do not adapt received-encrypted traffic at all.
* Do not log received-encrypted traffic.
* Do not cache received-encrypted traffic.
* Do not limit received-encrypted traffic bandwidth.
* Route received-encrypted traffic to a special cache peer.
* etc.


 In other news; HTTP is growing this thing called opportunistc security
 (aka. encryption). Where http:// schemed requests will be arriving over
 TLS, and maybe even going out via it and it traverses fully secure
 components. HTTPbis WG are already finding problems with servers not
 handling the schemes right in these conditions. Lets not add another
 bunch of bad assumptions to the mix. Follow the scheme.

 That seems like a counter-example to your own argument! The
 received_encrypted ACL should match if an http:// request arrived over
 TLS, and Squid may be able to offer better security for that
 transaction (e.g., by sending it to a Secure ICAP service or bypassing
 ICAP completely).


 Only if you misunderstand the security requirements around
 oppportunistic vs HTTPS.

I do not think so: The proposed ACL would help the admin to extend the
security opportunity to certain traffic. Whether to do that would
still be admin's choice (it is an ACL, not a hard-coded behavior!).
Without this ACL, making that decision becomes much harder because Squid
does not supply the necessary information in an easily accessible form.
That is why admins want this ACL!


 Opportunistic security (http:// over TLS) means if the admin is *able*
 and *willing* to pay the extra overhead costs of TLS, great. Otherwise
 there is no pressure to do so.

OK.

  - in these cases the proposed patch adds nothing.

Except it helps the admin

[squid-dev] [PATCH] received_encrypted ACL

2015-07-17 Thread Tsantilas Christos 

This patch adds received_encrypted ACL

The new received_encrypted ACL matches transactions where all HTTP 
messages were received over TLS or SSL transport connections, including 
messages received from ICAP servers.


Some eCAP services receive data from unencrypted sources. Some eCAP 
services are secure, but we assume that all are not secure until we 
add a configuration option to mark secure eCAP services.


Use case: Sending everything to Secure ICAP services increases 
adaptation performance overhead. Folks want to send received_encrypted 
transactions and only those transactions to Secure ICAP services.


NOTE: Currently there is not any mechanism to indicate if a cached 
object came from secure source or not, so we assume that all hits for 
secure requests  are secure too.


This is a Measurement Factory project.
Add received_encrypted ACL

The new received_encrypted ACL matches transactions where all HTTP
messages were received over TLS or SSL transport connections, including
messages received from ICAP servers.

Some eCAP services receive data from unencrypted sources. Some eCAP
services are secure, but we assume that all are not secure until we
add a configuration option to mark secure eCAP services.

This is a Measurement Factory project.

=== modified file 'src/AclRegs.cc'
--- src/AclRegs.cc	2015-04-10 08:54:13 +
+++ src/AclRegs.cc	2015-06-15 15:45:43 +
@@ -43,40 +43,41 @@
 #include acl/HierCode.h
 #include acl/HierCodeData.h
 #include acl/HttpHeaderData.h
 #include acl/HttpRepHeader.h
 #include acl/HttpReqHeader.h
 #include acl/HttpStatus.h
 #include acl/IntRange.h
 #include acl/Ip.h
 #include acl/LocalIp.h
 #include acl/LocalPort.h
 #include acl/MaxConnection.h
 #include acl/Method.h
 #include acl/MethodData.h
 #include acl/MyPortName.h
 #include acl/Note.h
 #include acl/NoteData.h
 #include acl/PeerName.h
 #include acl/Protocol.h
 #include acl/ProtocolData.h
 #include acl/Random.h
+#include acl/ReceivedEncrypted.h
 #include acl/Referer.h
 #include acl/RegexData.h
 #include acl/ReplyHeaderStrategy.h
 #include acl/ReplyMimeType.h
 #include acl/RequestHeaderStrategy.h
 #include acl/RequestMimeType.h
 #include acl/SourceAsn.h
 #include acl/SourceDomain.h
 #include acl/SourceIp.h
 #include acl/SquidError.h
 #include acl/SquidErrorData.h
 #if USE_OPENSSL
 #include acl/Certificate.h
 #include acl/CertificateData.h
 #include acl/ServerName.h
 #include acl/SslError.h
 #include acl/SslErrorData.h
 #endif
 #include acl/Strategised.h
 #include acl/Strategy.h
@@ -213,20 +214,22 @@
 ACL::Prototype ACLTag::RegistryProtoype(ACLTag::RegistryEntry_, tag);
 ACLStrategisedconst char * ACLTag::RegistryEntry_(new ACLStringData, ACLTagStrategy::Instance(), tag);
 
 ACL::Prototype Acl::AnyOf::RegistryProtoype(Acl::AnyOf::RegistryEntry_, any-of);
 Acl::AnyOf Acl::AnyOf::RegistryEntry_;
 
 ACL::Prototype Acl::AllOf::RegistryProtoype(Acl::AllOf::RegistryEntry_, all-of);
 Acl::AllOf Acl::AllOf::RegistryEntry_;
 
 ACL::Prototype ACLNote::RegistryProtoype(ACLNote::RegistryEntry_, note);
 ACLStrategisedHttpRequest * ACLNote::RegistryEntry_(new ACLNoteData, ACLNoteStrategy::Instance(), note);
 
 #if USE_ADAPTATION
 ACL::Prototype ACLAdaptationService::RegistryProtoype(ACLAdaptationService::RegistryEntry_, adaptation_service);
 ACLStrategisedconst char * ACLAdaptationService::RegistryEntry_(new ACLAdaptationServiceData, ACLAdaptationServiceStrategy::Instance(), adaptation_service);
 #endif
 
 ACL::Prototype ACLSquidError::RegistryProtoype(ACLSquidError::RegistryEntry_, squid_error);
 ACLStrategisederr_type ACLSquidError::RegistryEntry_(new ACLSquidErrorData, ACLSquidErrorStrategy::Instance(), squid_error);
 
+ACL::Prototype ACLReceivedEncrypted::RegistryProtoype(ACLReceivedEncrypted::RegistryEntry_, received_encrypted);
+ACLReceivedEncrypted ACLReceivedEncrypted::RegistryEntry_(received_encrypted);

=== modified file 'src/HttpMsg.cc'
--- src/HttpMsg.cc	2015-04-27 05:31:56 +
+++ src/HttpMsg.cc	2015-06-26 15:49:21 +
@@ -6,41 +6,42 @@
  * Please see the COPYING and CONTRIBUTORS files for details.
  */
 
 /* DEBUG: section 74HTTP Message */
 
 #include squid.h
 #include Debug.h
 #include HttpHeaderTools.h
 #include HttpMsg.h
 #include MemBuf.h
 #include mime_header.h
 #include profiler/Profiler.h
 #include SquidConfig.h
 
 HttpMsg::HttpMsg(http_hdr_owner_type owner):
 http_ver(Http::ProtocolVersion()),
 header(owner),
 cache_control(NULL),
 hdr_sz(0),
 content_length(0),
-pstate(psReadyToParseStartLine)
+pstate(psReadyToParseStartLine),
+sources(0)
 {}
 
 HttpMsg::~HttpMsg()
 {
 assert(!body_pipe);
 }
 
 HttpMsgParseState operator++ (HttpMsgParseState aState)
 {
 int tmp = (int)aState;
 aState = (HttpMsgParseState)(++tmp);
 return aState;
 }
 
 /* find end of headers */
 static int
 httpMsgIsolateHeaders(const char **parse_start, int l, const char **blk_start, const char **blk_end)
 {
 /*
  * parse_start points to the first line of HTTP message