On 20/06/2023 17:12, Dan McLaughlin wrote:
Mark,
What are your thoughts on changing the Tomcat codebase to return a 503
instead of a 404 if a context is marked as distributable or if
clustering is enabled and deployed but stopped? When I did searches
years ago on this issue, most people at the time would recommend
adding 404 to the fail_on_status, which is what we did...until I
realized that we were causing our own internal DOS attack when we had
a 404 mistakenly left in our apps; that got me thinking how easy it
would be to make mod_jk thrash by just requesting pages that didn't
exist. It's not a huge issue for us since most of our apps are
authenticated using SAML, so all requests are intercepted before the
request is ever sent to Tomcat, but for our apps that don't require
authentication, it would be easy to exploit any app that had 404 in
the fail_on_status.
I think the problem is the "STOPPED" state is used by different users
for different things. Some want it to be equivalent to "The application
isn't deployed" while others want it to be equivalent to "The
application is present but currently under maintenance". I don't think
we can safely infer which of those behaviors the user wants from the
clustering and/or distributable settings.
I think the best solution is the "maintenance in progress" servlet
deployed in the ROOT web application. Other options I considered:
1. New Lifecycle state "MAINTENANCE". This would be a significant change
and I don't think the size of the problem justifies the scale of the
changes required.
2. Extending/enhancing the "pause" feature. Not really the right place
to start as pausing a context doesn't allow it to be updated (assuming
updates are the main reason for the maintenance).
3. A per Host configuration option to set the status to be used for
deployed but stopped web applications. Defaults to 404. Could be
configured to be 503. Would require some changes to the mapper to
add/remove contexts on deploy/undeploy rather than start/stop. Actually,
this is a significant behavioural change since it changes the mapping.
And the rewrite valve may complicate things further.
The more I think about this, the more nervous I get about changes like
this introducing regressions.
I come back to the "maintenance in progress" servlet deployed in the
ROOT web application. The one use case this doesn't cover is maintenance
of the ROOT web application. Currently Tomcat is hard-coded to return a
404 if a request would be mapped to ROOT but that application isn't
started. I think a request to make that status configurable would be
implemented pretty quickly.
Mark
--
Thanks,
Dan
On Tue, Jun 20, 2023 at 10:41 AM Dan McLaughlin <d...@djabenterprises.com>
wrote:
We typically don't deploy a ROOT context in our production environments--for no
other reason than making it more difficult to poke around. I'll look at that
as an option. Thanks for the tips.
--
Thanks,
Dan
On Tue, Jun 20, 2023 at 10:28 AM Mark Thomas <ma...@apache.org> wrote:
On 20/06/2023 15:41, Dan McLaughlin wrote:
So I tried to create a Valve to check to see if the application is stopped
and convert the 404 response to a 503, but I haven't had any luck getting
it to work. Is there another internal API that I should be using?
context.getState().isAvailable
ways seems to report the app is available even though it's stopped.
The code is looking at the wrong Context. Since the web application has
been stopped the request won't be mapped to it. I'm guessing the request
has been mapped to the root context which is available.
You'll need to do something like:
Container[] containers = request.getHost().findChildren();
for (Container container : containers) {
if (container.getState().isAvailable()) {
continue;
}
Context context = (Context) container;
if (request.getDecodedRequestURI().equals(context.getPath()) ||
request.getDecodedRequestURI().startsWith(
context.getPath() + '/')) {
response.sendError(HttpServletResponse.SC_SERVICE_UNAVAILABLE);
}
}
I haven't optimised this at all. It isn't particularly efficient. It is
just to give you an idea.
Actually. I have just had a much better idea. It works by taking
advantage of the Servlet specification mapping rules which require the
longest context path match.
Lets assume you have /app1 /app2 and /app3
In your ROOT web application create a maintenance Servlet that just
returns a 503 and map it to "/app1/*" "/app2/*" and /app3/*".
If app1 is running, the longest context path match rule means it will be
mapped to /app1 and the application will handle it. If the web
application is stopped, the request will be mapped to ROOT where it will
match the maintenance Servlet and return a 503.
The only thing that this won't work for is if you want to take the RROT
web application out of service.
Mark
import org.apache.catalina.*;
import org.apache.catalina.connector.Request;
import org.apache.catalina.connector.Response;
import org.apache.catalina.valves.ValveBase;
import jakarta.servlet.ServletException;
import java.io.IOException;
import java.util.logging.Logger;
import java.util.logging.Level;
public class DownForMaintenanceValve extends ValveBase {
// Create a Logger
private static final Logger log = Logger.getLogger(DownForMaintenanceValve.
class.getName());
public DownForMaintenanceValve() {
log.info("DownForMaintenanceValve started");
}
@Override
public void invoke(Request request, Response response) throws
IOException, ServletException
{
Context context = request.getContext();
if (!context.getState().isAvailable()) {
log.info("Application is not available, sending 503");
response.sendError(503);
} else {
log.fine("Application is available, passing to next valve");
getNext().invoke(request, response);
}
}
}
--
Thanks,
Dan
On Wed, Jun 14, 2023 at 2:32 PM Mark Thomas <ma...@apache.org> wrote:
On 14/06/2023 19:49, Dan McLaughlin wrote:
Hello,
This is probably a question that would be better suited for the dev list,
but I thought I'd start here first.
That depends. It is generally better to start on the users list.
Does anyone understand the reasoning behind why Tomcat, when clustered,
throws an HTTP status 404 and not a 503 when you have an application
deployed but stopped or paused?
The issue you describe only affects stopped applications. If an
application is paused then any requests to that application should be
held until the application is unpaused (or the client timeouts out).
The current Tomcat Mapper dates back to at least Tomcat 4. It might be
earlier but I don't know the Tomcat 3 code well enough to find the
Tomcat 3 mapping code in the web interface and I'm not curious enough to
check the code out so I can use grep.
The clustering implementation dates back to Tomcat 5.
You'll need to dig through the archives to see if this topic was ever
raised and, if it was, the result of that discussion. Probably around
the time clustering was added.
I think I understand that my only option is to
failover for 404s considering the current implementation.
That might cause problems. If the node returning 404 is marked as down
you'll have a DoS vulnerability that is trivial to exploit.
I've looked to
see if there was a configuration setting related to clustering that would
allow me to change the behavior, and I couldn't find one; the only
solution
seems to be to write a custom listener that detects that an application
is
deployed but stopped or paused, and then throw a 503 instead.
That would be a better short-term solution and fairly simple to write.
I'd probably do it as a Valve as you'll get access to Tomcat's internals
that way.
The clustering implementation generally assumes that all applications
are available on all nodes. If that isn't the case I wouldn't be
surprised to see log messages indicating issues with replication.
What is the use case for stopping one (or more) web applications on a node?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
