Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Nils Breunese]

Christopher Schultz  wrote:

>> Well yeah, it’s not like Envoy is a super niche proxy. We also found
>> the exact same issue in two other proxies in our network by the way.
>> Any proxy that does not consider path parameters when doing
>> path-based access control will have this issue when combined with a
>> server that does support them.
> This statement can be generalized to the following:
> 
> "When HTTP proxies and origin-servers disagree about how to process
> requests (specifically their URLs), Bad Things can happen."
> 
> I would expect most proxies to behave the way that Envoy evidently does:
> pass-through the URL in the request to the origin server. It's kind of a
> requirement of the HTTP spec(s).

That’s also what I expected. The issue is not about the URL that is passed on, 
but the way path access rules are evaluated.

> On the other hand, if you are applying a security-constraint at the
> proxy layer, I would expect that the server would at least explain how
> path-parameters are handled/normalized/removed/resolved/etc. Either
> Envoy doesn't publish that information, or you didn't read that section
> of the manual.

We’ve been in contacted with the Envoy people as well. Currently Envoy has no 
concept of path parameters, which is why the combination of Envoy with path 
access rules and Tomcat is vulnerable. But it seems that based on this thread 
Envoy might start ignoring path parameters when doing path access checks.

> There may even be a setting like
> normalize-URLs-to-be-proxied-for-authorization-checks or something like
> that. Or maybe not.

They are already normalized for the check, but without considering path 
parameters, which makes the system vulnerable. The normalization is only for 
the access check. If deemed allowed the URL is passed on unmodified from the 
original request AFAIK.

> But again, if the proxy and origin disagree, you'd better know the
> details and plan for them.

So we’ve learned now, yes. 

Nils.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Christopher Schultz]

Nils,

On 9/24/20 13:29, Nils Breunese wrote:
> Christopher Schultz  wrote:
> 
>> On 9/24/20 07:46, Nils Breunese wrote:
>>> Mark Thomas  wrote:
>>>
 On 24/09/2020 11:02, Nils Breunese wrote:

 

> - Envoy allows the request based on the /v1/* rule, because it 
> does not support path parameters, because they are not part of
> any recent standard (RFC 2396 dropped them in 1998 [1])

 Envoy does support path parameters and is correctly doing so in this
 case as an HTTP reverse proxy by passing them through to the back-end.
 From an HTTP perspective "/bar" and "/foo/..;a=1/bar" are completely
 different URIs. Envoy's behaviour as an HTTP reverse proxy is correct.
>>>
>>> I wasn’t suggesting that Envoy should modify the URL that it passes 
>>> to the backend, but when doing its path-based access checks, I think
>>> it should ignore path parameters and normalize the path, otherwise it
>>> is trivial to get around these access restrictions.
>>>
>>> When only allowing /v1/* Envoy won’t allow /v1/../internal/secrets,
>>> because it normalises that to /internal/secrets and then sees it
>>> doesn’t mach the access rule, but it will allow
>>> /v1/..;color=red/internal/secrets because there is nothing to
>>> normalize as far as it’s concerned.
>> That seems inconsistent to me. If you remove the path parameter, you get
>> /v1/../internal/secrets which would be normalized to /internal/secrets
>> and therefore not allowed.
> 
> I’m not sure what you think is inconsistent. That is indeed what would happen 
> if Envoy would ignore the path parameter when doing the check, but it doesn’t 
> do that, because it doesn’t understand path parameters.
> 
>> But as a proxy, it should be forwarding the URLs as-is and may have
>> different normalization behavior in that case.
> 
> I guess we were indeed operating under the assumption that URLs are URLs, but 
> apparently different systems deal with them differently, which can cause 
> unexpected behaviour and even security vulnerabilities. This was news to us.
> 
>> Honestly, if you want Envoy to enforce this kind of protection, it is
>> Envoy you will need to configure properly. For example, if
>> path-parameters are not used in your environment, consider simply
>> rejecting any request that contains one and then your security layer
>> will work as (otherwise) expected.
> 
> This is indeed the workaround we decided to go with, but it feels weird that 
> we had to do this. I expect there are many other insecure setups around 
> because of the different ways URLs are handled by various systems.
> 
> If Envoy would ‘understand’ path parameters and that they should be 
> considered insignificant when doing path-based access checks, we would have 
> been fine. But it seems Envoy and Tomcat don’t agree on whether path 
> parameters are a thing or not. We’ll also talk to the Envoy people again 
> about this.
> 
>>> We currently have a policy to not secure the applications themselves
>>
>> Wow.
> 
> Ok, a little nuance: when it comes to path-based access restrictions.
> 
>>> , but handle this via generic access controls that can be centrally
>>> audited and configured independently of what language or server is
>>> used to implement the application.
>> Okay. It looks like those generic access controls need a significant
>> upgrade, then.
> 
> Well yeah, it’s not like Envoy is a super niche proxy. We also found
> the exact same issue in two other proxies in our network by the way.
> Any proxy that does not consider path parameters when doing
> path-based access control will have this issue when combined with a
> server that does support them.
This statement can be generalized to the following:

"When HTTP proxies and origin-servers disagree about how to process
requests (specifically their URLs), Bad Things can happen."

I would expect most proxies to behave the way that Envoy evidently does:
pass-through the URL in the request to the origin server. It's kind of a
requirement of the HTTP spec(s).

On the other hand, if you are applying a security-constraint at the
proxy layer, I would expect that the server would at least explain how
path-parameters are handled/normalized/removed/resolved/etc. Either
Envoy doesn't publish that information, or you didn't read that section
of the manual.

There may even be a setting like
normalize-URLs-to-be-proxied-for-authorization-checks or something like
that. Or maybe not.

Tomcat has a suite of settings in that same vein with all the defaults
being (a) (reasonably) spec-compliant and (b) safe.

But again, if the proxy and origin disagree, you'd better know the
details and plan for them.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Christopher Schultz]

Mark,

On 9/24/20 12:41, Mark Thomas wrote:
> On 24/09/2020 17:28, Christopher Schultz wrote:
> 
> 
> 
>> Tomcat will only use path parameters in the final segment of a URL e.g.
>> https://www.example.com/app/servlet;jsessionid=ABCD1234?q=search
> 
> Not quite. Tomcat will only *add* the jsessionid at the end but it will
> accept it on any segment.

Good point, but I would expect applications don't generally /move/ that
path parameter for any reason, so a deny rule for such things should
probably be both effective and otherwise benign.

> Internally, Tomcat has an API to access path parameters but it only
> tracks name and value (as that is all that is required to extract
> jsesisonid). It would be trivial to extend it to include path
> information as well.

I hadn't thought of that, but it's obvious when looking at the API. a
change to that API to make it "better" would probably be weird.
Something like this maybe:

URL: /a;x=1/b;y=2/c;z=2;q=4

request.getPathParameter("x") -> "1"
request.getPathParameters() -> [ x=1, y=2, z=2, q=4 ]
request.getPathParameters("/a") -> [ x=1 ]
request.getPathParameters("/a/b") -> [ y=2 ]
request.getPathParameters("/a/b/c") -> [ z=2, q=4 ]

>> Assuming your application doesn't use path-parameters for anything else,
>> you should be able to detect and block any non-terminal path-segment
>> which contains a parameter and simply refuse the request with 400 or
>> something similar.
> 
> That is probably the simplest option in this case.

It's what I would do if I (a) wanted to host secret and non-secret stuff
on the same backend server and (b) didn't feel like securing my
application(s).

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Nils Breunese]

Christopher Schultz  wrote:

> On 9/24/20 07:46, Nils Breunese wrote:
>> Mark Thomas  wrote:
>> 
>>> On 24/09/2020 11:02, Nils Breunese wrote:
>>> 
>>> 
>>> 
 - Envoy allows the request based on the /v1/* rule, because it 
 does not support path parameters, because they are not part of
 any recent standard (RFC 2396 dropped them in 1998 [1])
>>> 
>>> Envoy does support path parameters and is correctly doing so in this
>>> case as an HTTP reverse proxy by passing them through to the back-end.
>>> From an HTTP perspective "/bar" and "/foo/..;a=1/bar" are completely
>>> different URIs. Envoy's behaviour as an HTTP reverse proxy is correct.
>> 
>> I wasn’t suggesting that Envoy should modify the URL that it passes 
>> to the backend, but when doing its path-based access checks, I think
>> it should ignore path parameters and normalize the path, otherwise it
>> is trivial to get around these access restrictions.
>> 
>> When only allowing /v1/* Envoy won’t allow /v1/../internal/secrets,
>> because it normalises that to /internal/secrets and then sees it
>> doesn’t mach the access rule, but it will allow
>> /v1/..;color=red/internal/secrets because there is nothing to
>> normalize as far as it’s concerned.
> That seems inconsistent to me. If you remove the path parameter, you get
> /v1/../internal/secrets which would be normalized to /internal/secrets
> and therefore not allowed.

I’m not sure what you think is inconsistent. That is indeed what would happen 
if Envoy would ignore the path parameter when doing the check, but it doesn’t 
do that, because it doesn’t understand path parameters.

> But as a proxy, it should be forwarding the URLs as-is and may have
> different normalization behavior in that case.

I guess we were indeed operating under the assumption that URLs are URLs, but 
apparently different systems deal with them differently, which can cause 
unexpected behaviour and even security vulnerabilities. This was news to us.

> Honestly, if you want Envoy to enforce this kind of protection, it is
> Envoy you will need to configure properly. For example, if
> path-parameters are not used in your environment, consider simply
> rejecting any request that contains one and then your security layer
> will work as (otherwise) expected.

This is indeed the workaround we decided to go with, but it feels weird that we 
had to do this. I expect there are many other insecure setups around because of 
the different ways URLs are handled by various systems.

If Envoy would ‘understand’ path parameters and that they should be considered 
insignificant when doing path-based access checks, we would have been fine. But 
it seems Envoy and Tomcat don’t agree on whether path parameters are a thing or 
not. We’ll also talk to the Envoy people again about this.

>> We currently have a policy to not secure the applications themselves
> 
> Wow.

Ok, a little nuance: when it comes to path-based access restrictions.

>> , but handle this via generic access controls that can be centrally
>> audited and configured independently of what language or server is
>> used to implement the application.
> Okay. It looks like those generic access controls need a significant
> upgrade, then.

Well yeah, it’s not like Envoy is a super niche proxy. We also found the exact 
same issue in two other proxies in our network by the way. Any proxy that does 
not consider path parameters when doing path-based access control will have 
this issue when combined with a server that does support them.

Nils.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Mark Thomas]

On 24/09/2020 17:28, Christopher Schultz wrote:



> Tomcat will only use path parameters in the final segment of a URL e.g.
> https://www.example.com/app/servlet;jsessionid=ABCD1234?q=search

Not quite. Tomcat will only *add* the jsessionid at the end but it will
accept it on any segment.

Internally, Tomcat has an API to access path parameters but it only
tracks name and value (as that is all that is required to extract
jsesisonid). It would be trivial to extend it to include path
information as well.

> Assuming your application doesn't use path-parameters for anything else,
> you should be able to detect and block any non-terminal path-segment
> which contains a parameter and simply refuse the request with 400 or
> something similar.

That is probably the simplest option in this case.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Christopher Schultz]

Nils,

On 9/24/20 07:46, Nils Breunese wrote:
> Mark Thomas  wrote:
> 
>> On 24/09/2020 11:02, Nils Breunese wrote:
>>
>> 
>>
>>> - Envoy allows the request based on the /v1/* rule, because it 
>>> does not support path parameters, because they are not part of
>>> any recent standard (RFC 2396 dropped them in 1998 [1])
>> 
>> Envoy does support path parameters and is correctly doing so in this
>> case as an HTTP reverse proxy by passing them through to the back-end.
>> From an HTTP perspective "/bar" and "/foo/..;a=1/bar" are completely
>> different URIs. Envoy's behaviour as an HTTP reverse proxy is correct.
> 
> I wasn’t suggesting that Envoy should modify the URL that it passes 
> to the backend, but when doing its path-based access checks, I think
> it should ignore path parameters and normalize the path, otherwise it
> is trivial to get around these access restrictions.
> 
> When only allowing /v1/* Envoy won’t allow /v1/../internal/secrets,
> because it normalises that to /internal/secrets and then sees it
> doesn’t mach the access rule, but it will allow
> /v1/..;color=red/internal/secrets because there is nothing to
> normalize as far as it’s concerned.
That seems inconsistent to me. If you remove the path parameter, you get
/v1/../internal/secrets which would be normalized to /internal/secrets
and therefore not allowed.

But as a proxy, it should be forwarding the URLs as-is and may have
different normalization behavior in that case.

Honestly, if you want Envoy to enforce this kind of protection, it is
Envoy you will need to configure properly. For example, if
path-parameters are not used in your environment, consider simply
rejecting any request that contains one and then your security layer
will work as (otherwise) expected.

>> Clearly there is scope for more education and documentation on this
>> issue. The (very) short version is when using a reverse proxy in front
>> of Tomcat:
>> 1. Ideally don't rely on the reverse proxy to limit access to resources
>> on Tomcat. Secure Tomcat as if everything was accessible.
> 
> We currently have a policy to not secure the applications themselves

Wow.

> , but handle this via generic access controls that can be centrally
> audited and configured independently of what language or server is
> used to implement the application.
Okay. It looks like those generic access controls need a significant
upgrade, then.

>> 2. If you can't do 1, use mod_jk or the ISAPI redirector as they should
>> map URIs to proxy targets using the same mapping rules as Tomcat (and if
>> they don't we'll almost certainly treat that as a security issue).
> 
> Istio on Kubernetes uses Envoy as a proxy. I don’t think Envoy provides 
> something like this.
> 
>> 3. If you can't do 1 or 2 you can try and block potentially malicious
>> URIs in the reverse proxy but this is hard (need to think about "/../"
>> and "/./" sequences, path parameters, %nn encoding and combinations of
>> all three.
>>
>> If you get as far as option 3, you really need to go back and
>> re-consider option 1 or 2 as the chances of getting 3 100% right are slim.
> 
> AFAIK Envoy does normalize path traversal sequences like /../, but
> path parameters can break these sequences, and Envoy currently doesn’t
> ignore those, like the /v1/..;color=red/internal/secrets example.

Again, there might be a difference in normalization when it's being used
as a proxy.

Tomcat will only use path parameters in the final segment of a URL e.g.
https://www.example.com/app/servlet;jsessionid=ABCD1234?q=search

Assuming your application doesn't use path-parameters for anything else,
you should be able to detect and block any non-terminal path-segment
which contains a parameter and simply refuse the request with 400 or
something similar.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Nils Breunese]

Mark Thomas  wrote:

> On 24/09/2020 11:02, Nils Breunese wrote:
> 
> 
> 
>> - Envoy allows the request based on the /v1/* rule, because it does not 
>> support path parameters, because they are not part of any recent standard 
>> (RFC 2396 dropped them in 1998 [1])
> 
> Envoy does support path parameters and is correctly doing so in this
> case as an HTTP reverse proxy by passing them through to the back-end.
> From an HTTP perspective "/bar" and "/foo/..;a=1/bar" are completely
> different URIs. Envoy's behaviour as an HTTP reverse proxy is correct.

I wasn’t suggesting that Envoy should modify the URL that it passes to the 
backend, but when doing its path-based access checks, I think it should ignore 
path parameters and normalize the path, otherwise it is trivial to get around 
these access restrictions.

When only allowing /v1/* Envoy won’t allow /v1/../internal/secrets, because it 
normalises that to /internal/secrets and then sees it doesn’t mach the access 
rule, but it will allow /v1/..;color=red/internal/secrets because there is 
nothing to normalize as far as it’s concerned.

> Clearly there is scope for more education and documentation on this
> issue. The (very) short version is when using a reverse proxy in front
> of Tomcat:
> 1. Ideally don't rely on the reverse proxy to limit access to resources
> on Tomcat. Secure Tomcat as if everything was accessible.

We currently have a policy to not secure the applications themselves, but 
handle this via generic access controls that can be centrally audited and 
configured independently of what language or server is used to implement the 
application.

> 2. If you can't do 1, use mod_jk or the ISAPI redirector as they should
> map URIs to proxy targets using the same mapping rules as Tomcat (and if
> they don't we'll almost certainly treat that as a security issue).

Istio on Kubernetes uses Envoy as a proxy. I don’t think Envoy provides 
something like this.

> 3. If you can't do 1 or 2 you can try and block potentially malicious
> URIs in the reverse proxy but this is hard (need to think about "/../"
> and "/./" sequences, path parameters, %nn encoding and combinations of
> all three.
> 
> If you get as far as option 3, you really need to go back and
> re-consider option 1 or 2 as the chances of getting 3 100% right are slim.

AFAIK Envoy does normalize path traversal sequences like /../, but path 
parameters can break these sequences, and Envoy currently doesn’t ignore those, 
like the /v1/..;color=red/internal/secrets example.

Nils.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Nils Breunese]

Martin Grigorov  wrote:

> Someone else had the same/similar problem and the conclusion was that
> according to the Servlet specification this is the proper way to process
> the request - the request url should be normalized. If you need to protect
> some paths then you should do whatever is necessary in your application.

We have hundreds of applications running on Tomcat and path-based access 
control is currently handled outside Tomcat by Istio’s RBAC in the cloud. It 
appears that this is not a great match then.

> Please use secur...@tomcat.apache.org for reporting (possible) security
> problems in the future! Thanks!

I’m sorry. I read https://tomcat.apache.org/security.html 
 and it explicitly mentions using that 
address only for undisclosed security vulnerabilities. Since this issue seems 
to have been mentioned on the web in various places before, I thought it was 
fine to discuss this in public.

Nils.

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Nils Breunese]

Julian Reschke  wrote:

> Am 24.09.2020 um 12:02 schrieb Nils Breunese:
>> Hello,
>> 
>> I recently learned that when a server that supports path parameters [0] — 
>> like Tomcat (I found Jetty also does) — is run behind a reverse proxy that 
>> does path-based access control checks and does not support path parameters, 
>> your combined setup could be vulnerable.
>> 
>> Consider this setup:
>> 
>> 1. A Tomcat application without access restrictions
>> 2. An reverse proxy that only allows requests to /v1/* on the Tomcat 
>> application. I’ve used Envoy on Kubernetes, configured via Istio’s 
>> Role-Based Access Control (RBAC), but also observed this issue with other 
>> proxies.
>> 
>> Now I send a request for /v1/..;color=red/internal/secret to the proxy.
>> 
>> - Envoy allows the request based on the /v1/* rule, because it does not 
>> support path parameters, because they are not part of any recent standard 
>> (RFC 2396 dropped them in 1998 [1])
> > ...
> 
> That is very misleading.
> 
> RFC 2396 *extended* path parameters to be applicable to each path
> segment, see
> .
> 
> RFC 3986 further generalized things (see last paragraph of
> ).
> 
> They have not been "dropped".

Ok, interesting. On the Istio issue tracker [0] I got a response saying: “Path 
segments were part of RFC2396, but they are not part of RFC3986 which obsoleted 
it, so they are not part of any active standard for over 15 years.” [0]

If they are still part of current standards, then the argument could be made 
that any system that does path-based access checks should ignore path 
parameters when checking a path, right? Because otherwise it is trivial to get 
around such path-based access restrictions.

Nils.

[0] https://github.com/istio/istio/issues/27409#issuecomment-696784022
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Julian Reschke]

Am 24.09.2020 um 12:02 schrieb Nils Breunese:

Hello,

I recently learned that when a server that supports path parameters [0] — like 
Tomcat (I found Jetty also does) — is run behind a reverse proxy that does 
path-based access control checks and does not support path parameters, your 
combined setup could be vulnerable.

Consider this setup:

1. A Tomcat application without access restrictions
2. An reverse proxy that only allows requests to /v1/* on the Tomcat 
application. I’ve used Envoy on Kubernetes, configured via Istio’s Role-Based 
Access Control (RBAC), but also observed this issue with other proxies.

Now I send a request for /v1/..;color=red/internal/secret to the proxy.

- Envoy allows the request based on the /v1/* rule, because it does not support 
path parameters, because they are not part of any recent standard (RFC 2396 
dropped them in 1998 [1])

> ...

That is very misleading.

RFC 2396 *extended* path parameters to be applicable to each path
segment, see
.

RFC 3986 further generalized things (see last paragraph of
).

They have not been "dropped".

Best regards, Julian

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Mark Thomas]

On 24/09/2020 11:02, Nils Breunese wrote:



> - Envoy allows the request based on the /v1/* rule, because it does not 
> support path parameters, because they are not part of any recent standard 
> (RFC 2396 dropped them in 1998 [1])

Envoy does support path parameters and is correctly doing so in this
case as an HTTP reverse proxy by passing them through to the back-end.
>From an HTTP perspective "/bar" and "/foo/..;a=1/bar" are completely
different URIs. Envoy's behaviour as an HTTP reverse proxy is correct.

That the back-end may map URIs to resources in such a way that both
those URIs result in the same resource being served is solely a concern
for the back-end and entirely transparent to the HTTP reverse proxy.

The current RFC for URIs (3986) continues to make reference to
parameters within segments and using ';' as a delimiter for them. See
the final paragraph of section 3.3.
The reference in section D.2 to obsolete rules is a simplification of
the grammar, not a removal of features.

> - Tomcat parses ;color=red as a path parameter, resolves the rest of the path 
> to /v1/../internal/secret and consequently serves /internal/secret
> - I’m pretty sure that whoever configured that RBAC policy did not want this 
> to be possible

Then whoever configured the role-based access control shouldn't have
assumed that the reverse proxy used the same rules for URI to proxy
target mapping as the back-end used for mapping URIs to resources. They
either needed to ensure that Envoy was configured to map proxied
requests using the same mapping rules as the Servlet spec (I'd be
surprised if such a configuration setting was available) or to configure
the access control in the back-end.

This is a common error.

Since mod_jk and the the ISAPI redirector are provided by the Apache
Tomcat project it is reasonable for users to assume that they map URIs
to proxy targets using the same rules as the Servlet spec does for URIs
to resources. Where differences have been found over the years, CVEs
have been allocated.

If a generic HTTP reverse proxy (e.g. mod_proxy_http) is used then the
mapping of URIs to proxy targets is going to be HTTP based, not Servlet
based. That said, I understand that the httpd team is looking at
implementation a configuration option for mod_proxy that would cause it
to use Servlet mapping rules to map URIs to proxy targets.



> Would it be reasonable to create a ticket with a feature request for a flag 
> to disable path parameter support in Tomcat?

No. Support for path parameters is an explicit requirement of the
Servlet specification.

(Aside:
1. While the Servlet spec language isn't aligned with RFC 3986 the
expert group has made it clear that the intention is that references to
"path parameter" should be read as "parameter associated with any
segment of the URI".
2. Whether the Servlet spec should still be supporting URI re-writing as
a method of session tracking is debatable.
https://github.com/eclipse-ee4j/servlet-api/issues is the place to raise
the issue of removing it.
)

> Or an even more secure suggestion: make path parameter support an opt-in 
> feature, because it was dropped from the URI standard in 1998 and can make 
> setups like I described vulnerable?

No, because it hasn't been dropped from the URI standard and it is a
supporting path parameters is a mandatory requirement of the Servlet spec.

Clearly there is scope for more education and documentation on this
issue. The (very) short version is when using a reverse proxy in front
of Tomcat:
1. Ideally don't rely on the reverse proxy to limit access to resources
on Tomcat. Secure Tomcat as if everything was accessible.
2. If you can't do 1, use mod_jk or the ISAPI redirector as they should
map URIs to proxy targets using the same mapping rules as Tomcat (and if
they don't we'll almost certainly treat that as a security issue).
3. If you can't do 1 or 2 you can try and block potentially malicious
URIs in the reverse proxy but this is hard (need to think about "/../"
and "/./" sequences, path parameters, %nn encoding and combinations of
all three.

If you get as far as option 3, you really need to go back and
re-consider option 1 or 2 as the chances of getting 3 100% right are slim.

Finally, it is probably worth setting out what are the mapping rules
defined by the Servlet spec as the definition is very much implicit
rather than explicit.

1. Remove all path parameters
2. Collapse all "//" to "/"
3. Remove "/./" segments
4. Remove "/xxx/../" segments
   (If a leading "/../" remains that is an error)

Then Tomcat will map the resulting URI to a resource as per chapter 12
of the Servlet 4.0 spec.
mod_jk (and ISAPI) will map the resulting URI to the proxy targets as
per the docs
http://tomcat.apache.org/connectors-doc/reference/uriworkermap.html

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: 

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Martin Grigorov]

On Thu, Sep 24, 2020 at 2:11 PM Martin Grigorov 
wrote:

> Hi,
>
> On Thu, Sep 24, 2020 at 1:02 PM Nils Breunese  wrote:
>
>> Hello,
>>
>> I recently learned that when a server that supports path parameters [0] —
>> like Tomcat (I found Jetty also does) — is run behind a reverse proxy that
>> does path-based access control checks and does not support path parameters,
>> your combined setup could be vulnerable.
>>
>> Consider this setup:
>>
>> 1. A Tomcat application without access restrictions
>> 2. An reverse proxy that only allows requests to /v1/* on the Tomcat
>> application. I’ve used Envoy on Kubernetes, configured via Istio’s
>> Role-Based Access Control (RBAC), but also observed this issue with other
>> proxies.
>>
>> Now I send a request for /v1/..;color=red/internal/secret to the proxy.
>>
>> - Envoy allows the request based on the /v1/* rule, because it does not
>> support path parameters, because they are not part of any recent standard
>> (RFC 2396 dropped them in 1998 [1])
>> - Tomcat parses ;color=red as a path parameter, resolves the rest of the
>> path to /v1/../internal/secret and consequently serves /internal/secret
>> - I’m pretty sure that whoever configured that RBAC policy did not want
>> this to be possible
>>
>> Note that:
>> - The client can request any URL on the Tomcat server via this path
>> traversal attack.
>> - This also works with ‘empty’ path parameters, e.g.
>> /v1/..;/internal/secret
>> - The URL doesn’t necessarily need to contain ‘..’ for naughtiness to be
>> possible. When access to /public/secret would be restricted by the proxy,
>> but other /public/* paths would be allowed, then an attacker would still be
>> able to get the contents of /public/secret by requesting /public;/secret
>>
>> The fact that different servers handle paths with path parameters
>> differently seems to be known [2], but especially setups in which reverse
>> proxies don’t support path parameters, but the servers behind them do,
>> seems risky.
>>
>> Would it be reasonable to create a ticket with a feature request for a
>> flag to disable path parameter support in Tomcat? Or an even more secure
>> suggestion: make path parameter support an opt-in feature, because it was
>> dropped from the URI standard in 1998 and can make setups like I described
>> vulnerable?
>>
>
> I remember a similar discussion from a few months ago.
> Which version of Tomcat do you use ? Please try with the latest in case
> you use an older one.
>

I've just checked my mail archives.
Someone else had the same/similar problem and the conclusion was that
according to the Servlet specification this is the proper way to process
the request - the request url should be normalized. If you need to protect
some paths then you should do whatever is necessary in your application.
Please use secur...@tomcat.apache.org for reporting (possible) security
problems in the future! Thanks!

Martin


>
>
>>
>> Thanks, Nils.
>>
>> [0] RFC 1738 https://tools.ietf.org/html/rfc1738 <
>> https://tools.ietf.org/html/rfc1738> (1994) and RFC 1808
>> https://tools.ietf.org/html/rfc1808 
>> (1995).
>> [1] RFC 2396 G.4. Modifications from RFC 1808
>> https://tools.ietf.org/html/rfc2396 
>> (1998). Path parameters are also absent in RFC 3986
>> https://tools.ietf.org/html/rfc3986 
>> (2005).
>> [2]
>> https://i.blackhat.com/us-18/Wed-August-8/us-18-Orange-Tsai-Breaking-Parser-Logic-Take-Your-Path-Normalization-Off-And-Pop-0days-Out-2.pdf
>> <
>> https://i.blackhat.com/us-18/Wed-August-8/us-18-Orange-Tsai-Breaking-Parser-Logic-Take-Your-Path-Normalization-Off-And-Pop-0days-Out-2.pdf>
>> slides 41 and further
>
>

Re: Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Martin Grigorov]

Hi,

On Thu, Sep 24, 2020 at 1:02 PM Nils Breunese  wrote:

> Hello,
>
> I recently learned that when a server that supports path parameters [0] —
> like Tomcat (I found Jetty also does) — is run behind a reverse proxy that
> does path-based access control checks and does not support path parameters,
> your combined setup could be vulnerable.
>
> Consider this setup:
>
> 1. A Tomcat application without access restrictions
> 2. An reverse proxy that only allows requests to /v1/* on the Tomcat
> application. I’ve used Envoy on Kubernetes, configured via Istio’s
> Role-Based Access Control (RBAC), but also observed this issue with other
> proxies.
>
> Now I send a request for /v1/..;color=red/internal/secret to the proxy.
>
> - Envoy allows the request based on the /v1/* rule, because it does not
> support path parameters, because they are not part of any recent standard
> (RFC 2396 dropped them in 1998 [1])
> - Tomcat parses ;color=red as a path parameter, resolves the rest of the
> path to /v1/../internal/secret and consequently serves /internal/secret
> - I’m pretty sure that whoever configured that RBAC policy did not want
> this to be possible
>
> Note that:
> - The client can request any URL on the Tomcat server via this path
> traversal attack.
> - This also works with ‘empty’ path parameters, e.g.
> /v1/..;/internal/secret
> - The URL doesn’t necessarily need to contain ‘..’ for naughtiness to be
> possible. When access to /public/secret would be restricted by the proxy,
> but other /public/* paths would be allowed, then an attacker would still be
> able to get the contents of /public/secret by requesting /public;/secret
>
> The fact that different servers handle paths with path parameters
> differently seems to be known [2], but especially setups in which reverse
> proxies don’t support path parameters, but the servers behind them do,
> seems risky.
>
> Would it be reasonable to create a ticket with a feature request for a
> flag to disable path parameter support in Tomcat? Or an even more secure
> suggestion: make path parameter support an opt-in feature, because it was
> dropped from the URI standard in 1998 and can make setups like I described
> vulnerable?
>

I remember a similar discussion from a few months ago.
Which version of Tomcat do you use ? Please try with the latest in case you
use an older one.


>
> Thanks, Nils.
>
> [0] RFC 1738 https://tools.ietf.org/html/rfc1738 <
> https://tools.ietf.org/html/rfc1738> (1994) and RFC 1808
> https://tools.ietf.org/html/rfc1808 
> (1995).
> [1] RFC 2396 G.4. Modifications from RFC 1808
> https://tools.ietf.org/html/rfc2396 
> (1998). Path parameters are also absent in RFC 3986
> https://tools.ietf.org/html/rfc3986 
> (2005).
> [2]
> https://i.blackhat.com/us-18/Wed-August-8/us-18-Orange-Tsai-Breaking-Parser-Logic-Take-Your-Path-Normalization-Off-And-Pop-0days-Out-2.pdf
> <
> https://i.blackhat.com/us-18/Wed-August-8/us-18-Orange-Tsai-Breaking-Parser-Logic-Take-Your-Path-Normalization-Off-And-Pop-0days-Out-2.pdf>
> slides 41 and further

Tomcat's support for path parameters can expose resources despite reverse proxy access restrictions

[Nils Breunese]

Hello,

I recently learned that when a server that supports path parameters [0] — like 
Tomcat (I found Jetty also does) — is run behind a reverse proxy that does 
path-based access control checks and does not support path parameters, your 
combined setup could be vulnerable.

Consider this setup:

1. A Tomcat application without access restrictions
2. An reverse proxy that only allows requests to /v1/* on the Tomcat 
application. I’ve used Envoy on Kubernetes, configured via Istio’s Role-Based 
Access Control (RBAC), but also observed this issue with other proxies.

Now I send a request for /v1/..;color=red/internal/secret to the proxy.

- Envoy allows the request based on the /v1/* rule, because it does not support 
path parameters, because they are not part of any recent standard (RFC 2396 
dropped them in 1998 [1])
- Tomcat parses ;color=red as a path parameter, resolves the rest of the path 
to /v1/../internal/secret and consequently serves /internal/secret
- I’m pretty sure that whoever configured that RBAC policy did not want this to 
be possible

Note that:
- The client can request any URL on the Tomcat server via this path traversal 
attack.
- This also works with ‘empty’ path parameters, e.g. /v1/..;/internal/secret
- The URL doesn’t necessarily need to contain ‘..’ for naughtiness to be 
possible. When access to /public/secret would be restricted by the proxy, but 
other /public/* paths would be allowed, then an attacker would still be able to 
get the contents of /public/secret by requesting /public;/secret

The fact that different servers handle paths with path parameters differently 
seems to be known [2], but especially setups in which reverse proxies don’t 
support path parameters, but the servers behind them do, seems risky.

Would it be reasonable to create a ticket with a feature request for a flag to 
disable path parameter support in Tomcat? Or an even more secure suggestion: 
make path parameter support an opt-in feature, because it was dropped from the 
URI standard in 1998 and can make setups like I described vulnerable?

Thanks, Nils.

[0] RFC 1738 https://tools.ietf.org/html/rfc1738 
 (1994) and RFC 1808 
https://tools.ietf.org/html/rfc1808  
(1995).
[1] RFC 2396 G.4. Modifications from RFC 1808 
https://tools.ietf.org/html/rfc2396  
(1998). Path parameters are also absent in RFC 3986 
https://tools.ietf.org/html/rfc3986  
(2005).
[2] 
https://i.blackhat.com/us-18/Wed-August-8/us-18-Orange-Tsai-Breaking-Parser-Logic-Take-Your-Path-Normalization-Off-And-Pop-0days-Out-2.pdf
 

 slides 41 and further