from:"Berneburg, Cris J. \- US"

Terence

> I created an issue for the blank digest messages:
> https://issues.apache.org/jira/browse/INFRA-23675
> which appears to be due a missing CRLF sequence following
> the header section. It's currently "WAITING FOR INFRA" so
> I don't think anyone has had a chance to look at it.

Thanks for investigating and reporting the issue.  :-)  Glad to know the cause 
has been identified.

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: Fwd: users Digest 17 Aug 2022 09:26:06 -0000 Issue 14393 - "BLANK" DIGEST MESSAGE ATTACHMENTS

> 2. Also, some digest messages are blank for me, but other
> folks' replies to them are not.  It's often original messages
> from specific users.  Maybe we can compare what we see.
> Not using multiple client apps, I don't know if the blankness
> is due to client app misinterpretation or if the problem
> originates on the server.  I have not been keeping track of
> how long this has been happening, but it seems to be a
> "recent" issue, at least for me.  FYI, I use MS Outlook on Exchange Server.

Well, that's ironic.  :-)  My own messages in digest are blank!

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Fwd: users Digest 17 Aug 2022 09:26:06 -0000 Issue 14393 - "BLANK" DIGEST MESSAGE ATTACHMENTS

Hi Terence

I have similar issues.

> First, I was suddenly unable to send e-mail to the list using an
> e-mail address that I have used on the list since at least 2005,
> as mentioned above. I got around this by (re)subscribing to both
> users and users-digest. This may be why you found my e-mail
> address listed twice as a subscriber.

> What isn't clear is whether a subscription to the list in the non-
> digest form is now required to send messages to the list. (I was
> previously subscribed to the digest only and had been able to
> send messages to the list.) I should be able to test this without
> too much trouble.

1. I stopped being able to reply to the digest after being subscribed for a few 
years.  Thanks for the idea about subbing to the "users" messaging service.  At 
Mark's suggestion, I opened a Jira ticket, which is still unresolved, 
"Subscriber Reply Posts to users@tomcat.apache.org Bounced".  
https://issues.apache.org/jira/browse/INFRA-23619  I now see individual 
messages from myself since subscribing to that service (in addition to the 
digest).

> Second, some attachments in the digest are still not displayed
> in Thunderbird (shown as blank).I previously mistakenly reported
> that some digest attachments were not displayed in gmail but that
> looks to have been due to operator error as I'm now able to see
> attachments in gmail including those shown as blank in Thunderbird.

2. Also, some digest messages are blank for me, but other folks' replies to 
them are not.  It's often original messages from specific users.  Maybe we can 
compare what we see.  Not using multiple client apps, I don't know if the 
blankness is due to client app misinterpretation or if the problem originates 
on the server.  I have not been keeping track of how long this has been 
happening, but it seems to be a "recent" issue, at least for me.  FYI, I use MS 
Outlook on Exchange Server.

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [ANN] New committer: Han Li

Congrats Han!  :-D

--
Cris Berneburg
CACI Senior Software Engineer

-Original Message-
From: Mark Thomas 
Sent: Tuesday, September 6, 2022 3:38 AM
To: Tomcat Developers List ; Tomcat Users List 

Subject: [ANN] New committer: Han Li

Please join me in congratulating Han.

Kind regards,

Mark

This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Rename version 10.1 to 11

2022-03-18 Thread Berneburg, Cris J. - US

Chris, and the rest of the TC team

cs> Note that Java 10 will auto-migrate older applications for you
cs> without modification. It's kind of a friendly bootstrapping feature
cs> to help developers make the transition to pre-Jakarta-EE to
cs> port-Jakarta-EE.

Thaaanks!  :-)

cs> the transition from Java EE to Jakarta EE is going to be a big mess
cs> and the version-numbering for Tomcat is the last of anyone's
cs> problems. Aligning to the Jakarta EE version will help everybody
cs> moving forward, so that's what we've chosen to do.

+1

To quote Patrick Star, "Sounds reasonable."

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Odd EL resolution issue - java.lang.NoClassDefFoundError: package/Class1 (wrong name: package/class1)

2022-03-03 Thread Berneburg, Cris J. - US

Mark, et al

> Running Tomcat in a container via Docker Desktop on a Windows host
> with the web application served from a location on the host mounted
> /bound to the container is insecure.

So the app resides on the "host" OS file system and is mounted into the Docker 
"guest" container, rather than residing on a Docker volume or in the guest 
container's file system?

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: Tomcat 9 can not start on windows 10 as service

2022-02-10 Thread Berneburg, Cris J. - US

w> I install tomcat 9 using downloaded installation package. It was installed 
successfully.
w> I made tomcat manager working. I deployed my application... Suddenly, tomcat 
stopped.
w> Then I try to restart it using windows service. I got error 5: access 
denied. I uninstalled
w> tomcat and re-installed it. The same thing happened. Now I can go to 
tomcat\bin
w> directory run startup.bat. It works. What is wrong? How can I run it 
automatically using
w> windows service? Please.  Any information would be appreciated. Thanks in 
advance.

It sounds like you have done some good investigation so far.  Running manually 
as a check was a good idea.  :-)

Were you able to check the Windows Event Viewer to see if that had any clues?  
You would need to know the time when TC stopped working.  I'm not sure which 
category tab to check, application or system.

th> It sounds like the issue might not be Tomcat related.  Is any group policy,
th> like Software Restriction Policies in place in your company?

This also sounds like an interesting line of thought.  Perhaps there is 
software running that prevents "unapproved" software from running as a 
service...  What does your company IT department have to say about it?

- Cris B.




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: compression?

2021-08-10 Thread Berneburg, Cris J. - US

Hi Mark

crisb> P.S.: If a documentation update is recommended,
crisb> I would be happy to make the changes,
crisb> but I would probably need guidance for that too.  ;-)

markt> Source file is here:
markt> https://github.com/apache/tomcat/blob/main/webapps/docs/config/http.xml

markt> A pull request is fine.

Pull request #442 created on http.xml, "clarified compressionMinSize and 
compressibleMimeType".

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: compression?

2021-08-02 Thread Berneburg, Cris J. - US

Thanks Mark  :-)

crisb> Is it possible to connect IIS to TC using HTTP instead of AJP?
crisb> Several "Tomcat IIS How-To" articles all mention using AJP
crisb> (not HTTP) using an ISAPI redirector.

markt> In theory, yes. You'd need to find an HTTP reverse proxy component for 
IIS.
markt> This looks like the sort of thing you'd need:
markt> https://docs.microsoft.com/en-us/iis/extensions/url-rewrite-module/
markt> reverse-proxy-with-url-rewrite-v2-and-application-request-routing
markt> The downside is that you will need to manually configure a lot of the 
stuff
markt> AJP does "for free". Correctly configuring a reverse proxy is one of 
those
markt>  tasks where all sorts of things can catch you out.

Yeah, that looks way more complicated than what I was hoping for.  Talked with 
the sysadmin about it and he agreed, even though he implemented it on at least 
one of our dev servers.  We may roll that back in light of your suggestion 
below.

markt>  I'd probably look at getting IIS to compress the content instead:
markt>  
https://docs.microsoft.com/en-us/iis/extensions/iis-compression/iis-compression-overview

That looks better, much less complex and fragile.  I see in the 
" element in applicationHost.config" that you can specify 
mimeType's - perfect.  We'll see what the SA thinks.

The other option the SA and I talked about was dropping IIS altogether.  ;-)

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: compression?

2021-07-27 Thread Berneburg, Cris J. - US

Carsten and Mark

Thanks for the info.  :-)

crisb> Weird, when going thru IIS to TC, it's not compressed

c.klein> IIS fetches the requested resource from TC, acting as an HTTP client 
(or are you using AJP with IIS?).

markt> IIS will be using AJP to talk to Tomcat which doesn't support 
compression. You may be able to get IIS to compress the files.

Is it possible to connect IIS to TC using HTTP instead of AJP?  Several "Tomcat 
IIS How-To" articles all mention using AJP (not HTTP) using an ISAPI redirector.

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: compression?

2021-07-23 Thread Berneburg, Cris J. - US

Thanks Mark!

cb> 1. compressionMinSize - What are the units, bytes?
Markt> Yes.

cb> 2. compressibleMimeType - If you specify a type explicitly, [...]  Are [the 
defaults]
cb> over-ridden, so they need to be specified explicitly too?  Or is it 
cumulative?
Markt> Default is over-ridden.

OK, that worked when connecting directly to TC:

HTTP/1.1 200
vary: accept-encoding
Content-Encoding: gzip
Content-Type: application/json;charset=UTF-8
Content-Language: en-US
Transfer-Encoding: chunked
Date: Fri, 23 Jul 2021 16:37:48 GMT
Keep-Alive: timeout=20
Connection: keep-alive

Weird, when going thru IIS to TC, it's not compressed:

HTTP/1.1 200 200
Content-Type: application/json;charset=UTF-8
Content-Language: en-US
Server: Microsoft-IIS/10.0
Date: Fri, 23 Jul 2021 16:34:30 GMT
Content-Length: 3210105

cb> P.S.: If a documentation update is recommended, I would be happy to
cb> make the changes, but I would probably need guidance for that too.  ;-)

Markt> Source file is here:
Markt> https://github.com/apache/tomcat/blob/main/webapps/docs/config/http.xml

Markt> A pull request is fine. If you prefer to provide a patch, use "diff -u"
Markt> format, create a BZ issue and attach the patch.

I'll have a look at it later.  Also, I'm quite a newbie with git.

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: Log4j2 logging with Tomcat 9 web app

2021-07-21 Thread Berneburg, Cris J. - US

Hi Ravi  :-)

> My web app is based on Tomcat 9.0.45 server. I have migrated from Tomcat 7 to 
> Tomcat 9
> and from log4j 1.x to log4j 2.x.  I have updated the log4j2.properties as per 
> log4j 2.x standard,
> still my tomcat.log file is not getting generated and all the application log 
> are coming on
> console instead of redirecting this to tomcat.log file.  So
> 1.tomcat.log is not geting  generated
> 2. all the contents are logging and showing on the application console 
> instead of getting this
> logged inside the tomcat.log file.
> Tomcat 9.0.45 + log4j 2.14.1 is used. I am also attaching my log4j property 
> file.
> Please find this attached here.
> Kindly suggest me the solution.
> Thank you.

Thanks for supplying your "log4j2.properties" file.  (Actually, I kind of 
surprised the file was delivered intact, since the mailing list usually strips 
attachments.)  That properties file tells Log4j2 how to perform logging.  How 
did you tell Tomcat to use Log4j2?

FYI, I'm a little rusty as Log4j2.

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

compression?

2021-07-21 Thread Berneburg, Cris J. - US

Hi Folks :-)

Got some questions about turning on compression.  Looking at the documentation 
(I did not read the whole thing, just the portions in question), I still need 
some clarification.

https://tomcat.apache.org/tomcat-8.5-doc/config/http.html

1. compressionMinSize - What are the units, bytes?

2. compressibleMimeType - If you specify a type explicitly, like 
"application/json", what does it do with the defaults, like "text/html"?  Are 
they over-ridden, so they need to be specified explicitly too?  Or is it 
cumulative?

Thanks for your time.

P.S.: If a documentation update is recommended, I would be happy to make the 
changes, but I would probably need guidance for that too.  ;-)

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Strange error with JSP

2021-06-29 Thread Berneburg, Cris J. - US

Hi Chris

Was there a final resolution to this?

--
Cris Berneburg
CACI Senior Software Engineer

-Original Message-
From: Christopher Schultz 
Sent: Wednesday, June 2, 2021 1:52 PM
To: Tomcat Users List 
Subject: Strange error with JSP

All,

I don't do too much work with JSPs, but I do have a few quick-and-dirty 
administrative things including one called the "session snooper" which just 
dumps out loads of information about the current user's session object.

I'm getting this error in production, and I can reproduce it every time I 
access the page. Here's the exception stack trace:

java.lang.ClassNotFoundException: org.apache.jsp.admin.SessionSnooper_jsp
java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at
org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:128)
at
org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:59)
at
org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:159)
at
org.apache.jasper.servlet.JspServletWrapper.getServlet(JspServletWrapper.java:192)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:413)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:382)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:330)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
[...filters, etc...]

This is a relatively simple JSP. There are no tag libraries in use and there 
are 3 imports of JSPs which contain some static utility functions.

Both files
app/work/Catalina/localhost/[$context]/org/apache/jsp/admin/SessionSnooper_jsp.java
and
app/work/Catalina/localhost/[$context]/org/apache/jsp/admin/SessionSnooper_jsp.java
exist and have file-dates from way back in 2016. (No recent changes)

The context has been restarted/reloaded (not redeployed) recently using JMX a 
few times, but nothing else relevant comes to mind.

This is Tomcat 8.5.65 from a stock ASF-distrubuted tarball, launched using 
"catalina.sh start". Nothing fancy.

What other information can I collect to help debug this? My expectation would 
be that the class should be findable and runnable. Tomcat should not be 
tripping over its own feet on this one IMO.

Thanks,
-chris




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 500 instances of tomcat on the same server

2021-06-29 Thread Berneburg, Cris J. - US

Eric and Mark

Just curious...

Eric> We can run 75 to 125 instances of tomcat on a single Linux server

Eric, Do you have or need a centralized way of managing all those instances?  
It sounds like different support groups connect to their own instances, if I 
understand correctly.

Mark> if there are changes we could make to Tomcat that would it
Mark> easier to run and manage that many instances do let us know.
Mark> We'd be happy to consider them.

Mark, did you already have something in mind?  Like a TC Manager-manager?  Some 
sort of dashboard that is able to perform TC Manager ops against all the 
instances?

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Strange error with JSP

2021-06-07 Thread Berneburg, Cris J. - US

Chris

[major snippage]

CS> app/work/Catalina/localhost/[$context]/org/apache/jsp/admin/
CS> SessionSnooper_jsp.java exist and have file-dates from way back
CS> in 2016. (No recent changes)

CS> This is Tomcat 8.5.65 from a stock ASF-distrubuted tarball,
CS> launched using "catalina.sh start". Nothing fancy.

CS> org.apache.jasper.compiler.Compiler.isOutDated(Compiler.java:464)

*Something* must have changed, perhaps out of your control?  I vaguely remember 
a few years back a customer was having a problem with a page not loading due to 
a compile error.  The problem happened after we deployed a single-JSP fix at 
the same time the IT department changed the TC compiler or Java version.  The 
problem went away eventually, I'm guessing after the IT dept did another 
something.

Can you make a copy of the JSP and edit it in-place down to the bare minimum 
that it will still generate the exception?  That way the new copy of the code 
is freshly compiled and you have the possibility of narrowing things down?  
IOW, if you can't figure out what it is, figure out what it is not.

Is there a JAR file out of place?

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] web app big memory usage?

2021-06-03 Thread Berneburg, Cris J. - US

Thanks Chris

[snip, snip, snippety-snip]

CS> What's the database? And the driver?

Oracle 19, oracle.jdbc.OracleDriver - jdbc:oracle:thin.

CS> MySQL Connector/J used to (still does?) read 100% of the results
CS> into the heap before Statement.executeQuery() returns unless you
CS> specifically tell it not to. So if your query returns 1M rows, you
CS> might bust your heap.
CS> It's entirely possible that other drivers do similar things.

The JSON has all the rows, so it appears no pagination is being used on the DB 
level.

cb> Multiple TC instances
cb> (3) because multiple copies of the apps don't play nice with each
cb> other.  That is, we can't just rename the WAR files and expect the
cb> deployed apps to stay inside that context name (I think).

CS> You might want to look into that, eventually. If they aren't playing
CS> together nicely, they are not "good" servlet citizens. Solving those
CS> issues may improve other things. *shrug*

Yeah, I was working on that previously, but attention spans are short, and I 
got pulled off that task onto - SQUIRREL!

cb> StringBuilder - 264MB for the supporting byte array and 264MB for the
cb> returned String, about 790MB total for that piece of the pie.
cb> Contents were simply the JSON query results returned to the client.
cb> No mystery there.

Also, I noticed that the SB internal memory usage is about 2x the size of the 
actual contents.  Is that because each char is stored as 2 bytes for Unicode?  
(Not the char array to string conversion, which is different.)

CS> Yep: runaway string concatenation. This is a devolution of the
CS> "Connector/J reads the whole result set into memory before
CS> returning" thing I mentioned above. Most JSON endpoints
CS> return arbitrarily large JSON responses and most client
CS> applications just go "duh, read the JSON, then process it".
CS> If your JSON is big, well, then you need a lot of memory to
CS> store it all if that' who you do things.

Looking at the contents of the JSON, it's not normalized - a lot of redundant 
metadata.  Hand-editing the JSON for analysis reduced it from 135 MB to 26 MB.  
Maybe the code that generates it can be improved.

CS> If you want to deal with JSON at scale, you need to process
CS> it in a streaming fashion. The only library I know that can do
CS> streaming JSON is Noggit, which was developed for use with
CS> Solr (I think, maybe it came from elsewhere before that).
CS> Anyway, it's ... not for the faint of heart. But if you can figure
CS> out out, you can handle petabytes of JSON with a tiny heap.

I don't think we need to serve up that much data, but I'm guessing we can do 
better with what we do serve.  Interesting nonetheless.

CS> You might want to throttle/serialize queries you expect to
CS> have big responses so that only e.g. 2 of them can be running
CS> at a time. Maybe all is well when they come one-at-a-time,
CS> but if you try to handle 5 concurrent "big responses" you bust
CS> your heap.

Hmm... I had not thought of throttling that way, restricting the number of 
concurrent queries.  I was thinking about restricting the number of records 
returned.  Not sure how to handle lots of users connected but only a few able 
to query concurrently.  Different DB connection pool with fewer connections for 
queries?

cb> (At least StringBuilder is being
cb> used instead of plus-sign String concatenation.)

CS> In Java "..." + "..." uses a StringBuilder

I did not know that.  Or I forgot, in which case I can't tell the diff.  :-P

CS> In some code, "..." + "..." is just fine

Often it's run-on sentences of plus-sign concatenation with nested quotes, 
almost unreadable and even worse for editing.  I like to replace with SB for 
readability and maintainability.

CS>  I hate it when someone replaces it with:
CS>  String foo = new StringBuilder("bar").append("baz").toString();
CS>  because the compiler does the _exact same thing_ and you've
CS>  just made the code more difficult to read.

Ahhh, the classic train wreck.  :-)

CS>  in a *loop*, then replacing it with a StringBuilder is pretty
CS>  important for performance, otherwise the compiler will
CS>  do something stupid

I believe the technical term for that is "stoopid".  :-)  Yeah, I like to be 
strategic about SB's and loops.

CS>  You might actually have to start reading some code (shiver!).

"You're ... mocking me."  :-)  Actually, I might be able to pass it off onto 
the guy who wrote the library.  *phew*

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received

RE: Strange error with JSP

2021-06-03 Thread Berneburg, Cris J. - US

Hi Chris

cs> This is a relatively simple JSP. There are no tag libraries in use and
cs> there are 3 imports of JSPs which contain some static utility functions.

Sorry, no technical suggestions, but some questions...

Can you load those 3 dependent JSP's in a browser via URL (or are they 
protected)?  Are their class files there in the work folder?

Was a Java upgrade installed recently?

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: [OT] web app big memory usage?

2021-06-01 Thread Berneburg, Cris J. - US

Hi Chris

[lots of snippage]

cb> One of our web apps is using a "lot" of memory, specifically a big
cb> user query.  We'd like to find out why.
cb> 1. Is there a way to analyze uncollected garbage?
cb> * AWS EC2 instance.
cb> * There are other TC instances on the same server.
cb> * Each TC instance has multiple apps.

cs> What's the goal? Do you just Want To Know, or are you trying
cs> to solve an actual problem.

a. Barely enough memory to distribute among the multiple TC instances and the 
apps they support.  A big enough user query (no throttling) causes OOME's.  
Attempting to determine if the code is being wasteful in some way and therefore 
could be made more efficient.

b. It's a dev app server (EC2) which hosts diff stages in the dev process - 
dev, test, and prototype streams.  Multiple TC instances (3) because multiple 
copies of the apps don't play nice with each other.  That is, we can't just 
rename the WAR files and expect the deployed apps to stay inside that context 
name (I think).

c. I don't want to debug the code.  I'm relatively new to the project, 
unfamiliar with some of the code, and anticipate getting lost in the weeds.  
See point #1 below.  ;-)

cs> If you have a bunch of garbage that's not being cleaned up,
cs> usually it's because there is simply no need to do so. The GC
cs> is behaving according to the 3 laws of rob..., er, 3 virtues of
cs> computing[1]:
cs>
cs>1. Laziness: nothing needs that memory so... meh
cs>2. Impatience: gotta clean that Eden space quick
cs>3. Hubris: if I ever need more memory, I know where to find it

cs> [1] http://threevirtues.com/

Ha ha ha!  :-)

cs> How long does the query take to run?

Dunno about the time on the DB query itself.  From the user's point of view, a 
full minute plus.

cs> What kind of query is it? Are we talking about something like SQL

Yup.  Classic RDMS back-end.

cs> or some in-memory database or something which really does
cs> take a lot of memory for the application to fulfill the request?

Nah, nothing that fancy.  The only fancy part is using node.js for the 
front-end.

I followed Amit's and John's suggestion of using Eclipse Memory Analyzer Tool's 
"Keep unreachable options" when running a query from the app client.  Digging 
deeper into the Leak Suspects Report, I saw a StringBuilder - 264MB for the 
supporting byte array and 264MB for the returned String, about 790MB total for 
that piece of the pie.  Contents were simply the JSON query results returned to 
the client.  No mystery there.

I suspect that repeating the process with multiple queries will reveal multiple 
StringBuilder's each containing big honking JSON results.  So the issue may not 
be a problem with efficiency so much as one of simple memory hogging.  (At 
least StringBuilder is being used instead of plus-sign String concatenation.)

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: [OT] web app big memory usage?

2021-06-01 Thread Berneburg, Cris J. - US

Hi Raghunath

cb> One of our web apps is using a "lot" of memory,
cb> specifically a big user query.  We'd like to find out why.
cb> 1. Is there a way to analyze uncollected garbage?

rm> You could try using the Oracle utility - "jstat"  - for analyzing
rm> the GC in an active Java process (PID)
rm> The "gcold" option helps us to peep into the Old Generation area
rm> jstat -gcold PID
rm> jstat -gcoldcapacity PID
rm>
rm> https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jstat.html

That sounds interesting.  :-)  I'll look into it!

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] web app big memory usage?

2021-05-28 Thread Berneburg, Cris J. - US

Hi John :-)

cb> 1. Is there a way to analyze uncollected garbage?
cb> 2. Is that a reasonable way to identify potential memory usage problems?

jeg> MAT has an option  to "Keep unreachable options."  It's under preferences.

Thanks for the suggestion!  I did not know about that option.

jeg> It sounds like you don't have an actual leak, just high allocation/GC.

Yeah, that's what I think too.

jeg> My favorite tool for this is to use the Java Flight Recorder and analyze 
it with Java Mission Control.

Hmm... Sounds interesting.  I'll check it out!

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] web app big memory usage?

2021-05-28 Thread Berneburg, Cris J. - US

Hi Amit  :-)

cb> 1. Is there a way to analyze uncollected garbage?
cb> 2. Is that a reasonable way to identify potential memory usage problems?

ap> Have you enabled the " Enable 'keep unreachable objects'" setting of MAT?
ap> https://blog.gceasy.io/2015/12/11/eclipse-mat-titbits/

No, I had not heard of that before.  Thanks for the suggestion!

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[OT] web app big memory usage?

2021-05-27 Thread Berneburg, Cris J. - US

Hi Folks  :-)

One of our web apps is using a "lot" of memory, specifically a big user query.  
We'd like to find out why.

The Tomcat Web Application Manager Find leaks button said that "No web 
applications appear to have triggered a memory leak on stop, reload or 
undeploy."

Tomcat Manager Server Status shows that 1.7GB (82%) of G1 Old Gen space is 
being used that has not been recycled yet.

I grabbed a heap dump and used Eclipse Memory Analyzer, and it shows that only 
94MB of memory is being used when G1 Old Gen space used 1.8GB.  MAT seems to be 
looking only at the active objects, not the discarded ones.  IOW, we're looking 
at what the app is doing ATM, not what it already did.

I want to explore the 1.7GB garbage pile to see what's being thrown away, not 
what things are still being used, to determine wastefulness.

1. Is there a way to analyze uncollected garbage?

2. Is that a reasonable way to identify potential memory usage problems?

Some technical specifics:
* TC 8.5.63
* Java 1.8.0_291
* AWS EC2 instance.
* Windows Server 2016.
* Instance started as Windows Service.
* There are other TC instances on the same server.
* Each TC instance has multiple apps.

Thanks for reading this far.  :-)

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: temp folder?

2021-05-07 Thread Berneburg, Cris J. - US

Hi Mark

Thanks for getting back with me.  :-)

markt> What is the setting for unpackWARs for Host?

These are the host settings in server.xml:

name="localhost"
appBase="webapps"
unpackWARs="true"
autoDeploy="true"
deployOnStartup="false"

markt> Running directly from a WAR (with unpackWARs="false"
markt> file will impact performance. It looks as if something
markt> is unpacking the WAR to the temp directory.

Where is it supposed to unpack the WAR files to?  I would have thought the work 
folder.

markt> Tomcat does provide the org.apache.catalina.webresources.
markt> ExtractingRoot resources implementation to help alleviate
markt> performance issues in this case but that should only
markt> extract the JARs in WEB-INF/lib and location they are
markt> extracted to should be under the work directory and include
markt> "application-jars" in the path.

OK good to know that at least for JAR's the "normal" place is the work folder 
and *not* the temp folder.

markt> Maybe some custom "unpack to temp" code?

That's what I'm afraid of.  :-\  What's weird(er) is that the default TC apps 
like docs and manager are copied to the temp folder too.  Also, the subfolders 
start with a number, like "0-app1", "4-docs", and "5-manager".  Does that 
provide a clue, or is that just normal?

Could the destination for unpacking the WAR files be changed from default with 
a setting or an environment variable?

--
Cris Berneburg
CACI Senior Software Engineer

-Original Message-

cb> Sometimes we get strange errors after deployments to our
cb> test server.  We just "solved" some weirdness by manually
cb> cleaning out the TC temp folder(s) - again.

cb> Looking in our TC temp folder, I see subfolders that match
cb> all the webapps [...] Looking in a subfolder, like temp/
cb> 3-app4, it appears to be an exact copy of everything in the
cb> webapps/app4 folder, which is just the extracted app4.war
cb> file. [...] The temp/app4 folder does not seem to contain
cb> temporary files, like output files for Excel reports, etc.
cb> Same for the other subfolders.  Is that normal?

cb> I see references to the temp folder in tomcat8-stdout.x.log
cb> [...] Why is it trying to access files in the temp subfolder
cb> instead of the webapps subfolder?  (Looks like I have some
cb> app debugging to do?)




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

temp folder?

2021-05-05 Thread Berneburg, Cris J. - US

Hi Folks

Sometimes we get strange errors after deployments to our test server.  We just 
"solved" some weirdness by manually cleaning out the TC temp folder(s) - again.

Googling confirms what I thought about the TC work versus temp folder:
* "work stores compiled JSPs and other assets".
* "temp is used to store files created using the Java File API for creating 
temporary files".

Looking in our TC temp folder, I see subfolders that match all the webapps 
(some names changed to protect the not-so-innocent):
* 0-app1
* 1-app2
* 2-app3
* 3-app4
* 4-docs
* 5-manager
* 6-trap

Looking in a subfolder, like temp/3-app4, it appears to be an exact copy of 
everything in the webapps/app4 folder, which is just the extracted app4.war 
file.  (The webapps folder has a copy of app4.war.)  The temp/app4 folder does 
not seem to contain temporary files, like output files for Excel reports, etc.  
Same for the other subfolders.  Is that normal?

Some technical specifics:
* TC 8.5.63
* Java 1.8.0_291
* AWS EC2 instance.
* Windows Server 2016.
* Instance started as Windows Service.
* -Dcatalina.home=D:\Tomcat8_1
* -Dcatalina.base=D:\Tomcat8_1
* -Djava.io.tmpdir=D:\Tomcat8_1\temp
* There are other TC instances on the same server.
* Each TC instance has multiple apps.

I see references to the temp folder in tomcat8-stdout.x.log  Below are some 
excerpts.  Why is it trying to access files in the temp subfolder instead of 
the webapps subfolder?  (Looks like I have some app debugging to do?)

* 2021-05-05 07:03:38,383 DEBUG [localhost-startStop-1] (?:?) - Attempting to 
obtain an input stream to 
file:/D:/Tomcat8_1/temp/0-app1/WEB-INF/classes/action.properties.

* 2021-05-05 07:04:52,426 localhost-startStop-1 DEBUG Apache Log4j Core 2.12.1 
initializing configuration 
XmlConfiguration[location=D:\Tomcat8_1\temp\1-app2\WEB-INF\classes\log4j2.xml]

* 07:04:53.990 [localhost-startStop-1] DEBUG 
org.springframework.context.annotation.ClassPathBeanDefinitionScanner - 
Identified candidate component class: file 
[D:\Tomcat8_1\temp\1-app2\WEB-INF\classes\app\HelloWorld.class]

* 2021-05-05 07:05:10,007 DEBUG [localhost-startStop-1] (?:?) - Attempting to 
obtain an input stream to 
file:/D:/Tomcat8_1/temp/2-app3/WEB-INF/classes/action.properties.

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: [OT?] caching DB items in startup listener

2021-04-16 Thread Berneburg, Cris J. - US

Hi Chris

cb> I was thinking of a servlet request (or something) that is called on
cb> startup that could also be called later on-demand(?).

cs> How would you trigger that servlet to be called on startup?
cs> Some kind of script that does catalina.sh && sleep $time
cs> && curl http://example.com/load-stuff ??
cs> How would you determine the value of $time? What if it fails?

Pfft, beats me.  :-)  I was just grasping at straws, apparently.

cs> You can also use thread-safe classes which either implement
cs> their thread-safety in one of a few different ways, synchronized
cs> blocks being one of those strategies.

Got any buzzwords for me that I can lookup "one of a few different ways", other 
than synchronized blocks?

cs> "freshen" the data from the database if it had been altered by
cs> some other process e.g. an update from a database where new
cs> content is added, then migrated into production via direct SQL
cs> drop. So it really was a "reload" operation. These days, it's an
cs> "unload" operation. :)

I was wondering about that.  Sounds like it basically invalidates the cache so 
it can be reloaded later when needed.

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT?] caching DB items in startup listener

2021-04-14 Thread Berneburg, Cris J. - US

Hi Thomas

Thanks for the info and your opinion!  :-)

cb> 1.  Is performing DB heavy-lifting operations in ServletContextListener
cb> a "reasonable" practice?
cb> 2.  Is there a "better" way of caching said items at application
cb> startup?

tm> What happens when the DB has problems when the webapp starts?
tm> Will the startup fail then?

Good question.  I don't know, but I would guess it would fail.  Or the web app 
would be in an unusable state since the needed cache would be empty.  I think 
at that point the app would need to be restarted.

tm> I think doing lazy init is the better approach

I'm starting to agree.  :-)

tm> when db comes back it will work again after the webapp did start.

So the web app would be more "robust" - it would cache the data when the DB is 
back online.  Sounds good.  :-)

Hmm... I'm kind of undecided about this.  If the DB is down during startup, 
then the web app would be unusable anyway.  Would it not make sense for the app 
to be down too?  I guess it depends on how the app handles DB connections and 
errors?  If the app:
* Displays a generic "System is down for maintenance" message when the DB is 
inaccessible and prevents the user from clicking things.
* Versus displaying weird messages to the user whenever they click a button or 
link but does not stop them from trying.

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT?] caching DB items in startup listener

2021-04-14 Thread Berneburg, Cris J. - US

Hey Chris

cb> 1.  Is performing DB heavy-lifting operations in ServletContextListener a 
"reasonable" practice?
cb> 2.  Is there a "better" way of caching said items at application startup?

cs> IMHO there is no better way than using a ServletContextListener to load 
things at startup.

OK, good to know that using SCL is "reasonable".

cs> Your only other spec-compliant option is to use a Servlet with 
load-on-startup set
cs> and do your work in the init() method, which is ... ugly.

I was thinking of a servlet request (or something) that is called on startup 
that could also be called later on-demand(?).

cs> Another option would be to perform "lazy loading" instead of a-priori 
loading of this data.
cs> You will take the hit of loading the data when it is first requested, which 
may negatively
cs> impact user experience. It might also mean that you have to be more careful 
about
cs> cross-thread synchronization, etc. since you can't guarantee that the work 
has already
cs> been done before a client tried to access the cache.

cs> If you are concerned about startup times, lazy-loading is a good solution.
cs> It can also improve your memory usage if that data is never actually needed.

+1.  I like this.  "Smarter" caching.  Only load the data you need when you 
need it.

cs> We have a primary application at $work where we need to have a lot of 
information
cs> in mrmoey to be able to do important stuff.  [...] We loaded 100% of it 
every time at startup.
cs> [...] I switched to loading things on-demand and it made not only a 
significant performance
cs>  improvement on startup [...] it significantly reduced the memory footprint 
of the
cs> in-memory cache of data

How were you "careful about cross-thread synchronization", synchronized blocks?

cs> We also have a user-initiatable process to "reload" the data

Where do you do the loading and reloading, a in a servlet request?

cs> [Now] it just empties the cache and does nothing else. More faster. :)

"More faster"  :-)

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[OT?] caching DB items in startup listener

2021-04-08 Thread Berneburg, Cris J. - US

Hi Folks

I'm working on an old legacy app and noticed something.  It caches a bunch of 
info (lookup table data) from the database using a ServletContextListener.  I 
think opening DB connections in a listener is reasonable.  While there is no 
business logic in the listener, I'm not sure doing a bunch of DB heavy-lifting 
operations in a context listener is a "good thing", although I don't really 
have a concrete reason why.  Perhaps I'm just being fussy.

Anyway, in your opinion:

  1.  Is performing DB heavy-lifting operations in ServletContextListener a 
"reasonable" practice?
  2.  Is there a "better" way of caching said items at application startup?

Thanks for your time and consideration.  :-)

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: Low throughput with HTTP2

2020-09-25 Thread Berneburg, Cris J. - US

Thanks again Mark  :-)

mt> how that Map is pruned (it is currently too aggressive)

mt> if Tomcat is processing 10k req/s just keeping track of
mt> the last 30s is potentially 300k streams. How to do that
mt> efficiently for all usage patterns is a problem that
mt> needs some thought.

Sounds a bit like garbage collection.  Is aging part of the process - a 
map/queue combo?

cjb> How could the closed stream footprint be reduced?
cjb> Could the structure holding a closed stream:
cjb> a. Be replaced with a smaller one?
cjb> c. Or did you already have something in mind?

mt> A form of a). I'm looking at this now.

cjb> b. De-reference other objects no longer needed?
cjb> Hmm... that might lead to NPE's and thus unnecessary
cjb> null checking.

mt> Tried that. Lots of NPE regressions to the point that
mt> I reverted the change to look for a better solution.

Hey great, I'm beginning to understand!  :-D

mt> we have all the plumbing to correctly determine
mt> relative priority [...] we don't use it to prioritise
mt> streams when flow control windows are not an issue

mt> I started to look at this a while ago but it gets very
mt> complex quite quickly. It would be simpler if we were
mt> just serving static content.

Ha ha, httpd!  Hang on, does httpd handle a similar situation too?

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Low throughput with HTTP2

2020-09-23 Thread Berneburg, Cris J. - US

Hi Mark

Thanks for taking the time to explain that to me.  :-)

A few more questions, if you don't mind.

cjb> TC thinks the stream should be closed when the client
cjb> thinks the stream is still open?  Basically RST_STREAM
cjb> is a keep-alive?

mt> No. The stream closed cleanly. The client is sending
mt> RST_STREAM due to what is suspected to be a client bug.
mt> RFC 7540 says the server must ignore such frames and can,
mt> if a frame is received a significant period after the
mt> stream closed, treat it as a protocol error (and close
mt> the connection).

mt> Separately, the server should (as per the RFC) retain
mt> state for closed streams to support prioritisation.

mt> Currently Tomcat uses a single Map to track the state of
mt> closed streams for priority and to identify streams have
mt> been closed for an *in*significant amount of time.

mt> The issues immediately at hand are:
mt> - how that Map is pruned (it is currently too aggressive)

What would you consider "less aggressive"?  Would aggressiveness depend on 
system load?

mt> - that under high load a "significant period" becomes a
mt>   few milliseconds

Sounds like "significant period" varies depending on system load.

mt> currently memory footprint of a closed stream is much
mt> larger than it needs to be

How could the closed stream footprint be reduced?  Could the structure holding 
a closed stream:

a. Be replaced with a smaller one?

b. De-reference other objects no longer needed?  Hmm... that might lead to 
NPE's and thus unnecessary null checking.

c. Or did you already have something in mind?

mt> while we have all the plumbing to correctly determine
mt> relative priority and use it when allocating window
mt> updates in the case where the connection flow control
mt> window is smaller than the total data the streams want
mt> to send - we don't use it to prioritise streams when
mt> flow control windows are not an issue

Is that an FYI or a to-do?

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Low throughput with HTTP2

2020-09-22 Thread Berneburg, Cris J. - US

Hi Mark

As with most topics here, I struggle to understand what is being discussed.  
:-)  So please bear with me.

> improving how Tomcat handles traffic like this.
>
> Looks like Tomcat could prune the closed streams
> less aggressively.
>
> At the moment it waits until there are
> maxConcurrentStreams + 10% in the map and then:
> - removes all closed streams without children
> - [snip] with children [snip]
> - [snip] closed final streams [snip]
>
> [snip] the size of the map increases to ~110 and
> then drops to ~5, increases to ~110 and repeats.
>
> I'm currently thinking about different pruning
> strategies. The associated memory footprint is
> also part of my thinking.

TC thinks the stream should be closed when the client thinks the stream is 
still open?  Basically RST_STREAM is a keep-alive?

So a passenger (client) discharges from a taxi and pays the driver (server), 
but asks the driver to wait (RST_STREAM), so the meter (stream) is still 
running.  How long does the driver wait (timeout) before driving away?  Does 
the driver honk the horn (send a wake-up packet) before looking for a new 
customer?

Is the issue a matter of "how" or "when"?  If TC receives RST_STREAM then 
restart the timeout clock.  To prevent abuse allow a limited number of 
successive keep-alive frames.  If a certain number of RST_STREAM's are 
received, aka threshold is reached, with nothing else occurring, then close the 
stream.  That could be configurable.

How about instead of a binary state of open or closed the state is trinary - 
open, stale, closed?
- Open, don't prune.
- Closed, prune.
- Stale:
  a. Move to closed after timeout or too many RST_STREAM's.
  b. Consider open if receive useful traffic.

Also, if there are multiple pruning strategies, allow a single method to be 
selected per connector config or for the whole TC instance.

I hope this is helpful.  If not, well, maybe at least it's educational.  ;-)

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

RE: [OT] RE: How to get the tag name from within a taglib class ?

2020-09-15 Thread Berneburg, Cris J. - US

Thanks Chris!

CS> IMO, the JSP effort was a stepping-stone on a path to better
CS> technologies like Velocity, FreeMarker, and others. If I were
CS> king, JSP would just go away. Just my POV of course [...]

cjb> what do you like better about Velocity, FreeMarker, etc.
cjb> more than JSP?

CS> I started using Velocity years ago [...] It definitely has its
CS> warts but it's relatively actively maintained, and anything I
CS> need I can get in and do myself, submit patches, etc.
CS>
CS> Advantages over JSP (IMHO):
CS>
CS> - Can't execute direct Java code, ever
CS> - Non-verbose syntax
CS> - No limit on template length [...]
CS> - Easy to install POJO "tools" which just expose Java objects
CS>   to the runtime so you can $tool.doSomething() [...]
CS> - Can load templates from anywhere (disk, DB, URL, etc.)

Good to know!  I also see that it is an ASF project.

Is Velocity interpreted or compiled like JSP?  I'm thinking of performance 
impacts, like during loops.

Answering my own question, the Velocity FAQ says, "Velocity doesn't compile 
your templates. They are parsed into an AST (abstract syntax tree) and that is 
what gets cached."

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

[OT] RE: How to get the tag name from within a taglib class ?

2020-09-14 Thread Berneburg, Cris J. - US

Hey Chris

CS> IMO, the JSP effort was a stepping-stone on a path to better
CS> technologies like Velocity, FreeMarker, and others. If I were
CS> king, JSP would just go away. Just my POV of course, you are
CS> welcome to fall in love with JSP. :)

Seeing as I am ever on the trailing edge of learning new or even dated 
technologies, what do you like better about Velocity, FreeMarker, etc. more 
than JSP?

--
Cris "NOT Trying to Start a Flame War" Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: How to get the tag name from within a taglib class ?

2020-09-14 Thread Berneburg, Cris J. - US

Rony

RF> If possible I would like to write a single tagclass, but use it
RF> for two or more different tags, as the implementation would share
RF> quite a lot of code. Besides, it might be helpful for debugging.

CS> Feel free to build a base class with the shared code and then implement
CS> the differences in subclasses.

Sure, implementing a sub-class of TagSupport that acts as a custom tag 
super-class works fine.

RF> I would have a need to find out the tag name
RF> that caused the tagclass to run.
RF>
RF> Is this possible? If so, how would one be able to get at
RF> that tag name (any brief hints would suffice) ?

Not sure exactly what you mean...

Here's an example:

- CustomTagImplementation.java contains:
public class CustomTagImplementation extends TagSupport

- tags.tld contains:

CustomTag1
tags.CustomTagImplementation


CustomTag2
tags.CustomTagImplementation


- page.jsp contains:




Are either of these what you mean?

A. Get surrounding tag "" - TagSupport/CustomTagImplementation/getParent(), 
for calculating something like an xpath?

B. Get tag definition name "CustomTag1" - Ouch.  Sorry, no help there.  How 
about an intellectual exercise to kill some time?  :-)

Suppose you had a map of classes and associated tag names from either parsing 
tags.tld directly or exposing whatever structure holds classes instantiated 
from it.  You might still have a lookup problem due to a one-to-many 
relationship.  Using CustomTagImplementation/getClassName() as the map lookup 
key would have 2 theoretical answers, both "CustomTag1" and "CustomTag2".

By the time the page code is executed, the JSP has aleady been compiled.  
Looking at pre-compiled org/apache/jsp/page_jsp.java, each call to 
 gets its own _jspx_meth_* method.  There is a comment in 
each method, "//  tags:CustomTag1".  Seems kinda messy in there, and 
potentially fragile to depend on the pre-compiler output format.

It's too bad TagSupport is not auto-magically fed as a parameter the name of 
tag definition in the page that "called" it.  All in the engine, no changes to 
JSP pages.

While I have not used it myself, have you looked at SimpleTagSupport to see if 
it has something useful?

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Security audit raises questions (Tomcat 7.0.93)

2020-03-18 Thread Berneburg, Cris J. - US

Hi JHHL

> security audit on the Tomcat server we maintain

My condolences.  :-)  We're gone through several scans over the past couple 
years too.  Yeah, it's a pain.

If you can get the report details, it may provide enough info to pinpoint the 
exact problems.  Checkmarx scanning software does, I think.

Also, a strategy I found helpful was to reduce the "attack surface".  Get rid 
of anything flagged that you don't use rather than trying to fix the issues.

> First, it found a cross-site scripting vulnerability.

For scans of our systems, the XSS vulnerabilities were poorly protected JSP 
expression language, uh, expressions.  :-)  Using standard tag libraries to 
wrap ${expressions} helped.  Also, defining a custom sanitize function used in 
JSP pages like ${fn:escapeXml(param.xxx)} satisfied requirements in the 
negotiation process.

Something we did not get around to was moving the JSP files to the 
WebContent\WEB-INF folder so they could not be called directly with injected 
malicious parameters.

> Second, it found the HTTP DELETE method enabled.

Do you need it?  Can you disable it?

> Fourth, it found the HTTP OPTIONS method enabled.

Again, do you need it?  Can you disable it?

> the click-jacking vulnerability came up [...] just now set up
> the filter and filter-mapping in conf/web.xml, so that is
> hopefully taken care of in the next restart.

+1  :-)

--
Cris Berneburg
CACI Lead Software Engineer

-Original Message-
From: James H. H. Lampert 
Sent: Tuesday, March 17, 2020 6:05 PM
To: Tomcat Users List 
Subject: Security audit raises questions (Tomcat 7.0.93)

Ladies and Gentlemen:

One of our customers did a security audit on the Tomcat server we maintain on 
their system, and it found a few issues:

First, it found a cross-site scripting vulnerability.

Second, it found the HTTP DELETE method enabled.

Third, it found a click-jacking vulnerability.

Fourth, it found the HTTP OPTIONS method enabled.

Back in October, the click-jacking vulnerability came up on another customer 
box; I've found the thread, and just now set up the filter and filter-mapping 
in conf/web.xml, so that is hopefully taken care of in the next restart.

But I have no idea what to do about the cross-site scripting vulnerability, or 
the DELETE and OPTIONS methods, and I'm having trouble understanding the 
materials I've found.

--
JHHL




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] TLSv1.3 in TC8.5 + Azul Java 8

2019-08-06 Thread Berneburg, Cris J. - US

-Original Message-
From: Christopher Schultz  

> "things to look into when I retire and my house is totally clean and
> my kids are finally out of the house" so of course, I'll never get around to 
> it.

+1  :-)

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Security vulnerabilities with tomcat 9

2019-07-24 Thread Berneburg, Cris J. - US

Hi Sumit

Please see my response below your question.

-Original Message-
From: Sumit Bhardwaj  
Sent: Saturday, July 20, 2019 8:48 AM
To: Tomcat Users List 
Subject: Security vulnerabilities with tomcat 9

> Hi,
>
> We are using tomcat 9 and getting following two vulnerabilities in security 
> scans.
>
> Cookie Does Not Contain The "secure" Attribute (1)  Cookie Does Not Contain 
> The "HTTPOnly" Attribute (1)
>
> We have done things mentioned in
> https://geekflare.com/secure-cookie-flag-in-tomcat/
>
> 
> true
> true
> 
>
> and also updating the *context.xml for *useHttpOnly="true"
> It has not helped.
>
> We also tried updating our web application's web.xml with the cookie-config, 
> but it has also not helped.
>
> What else do we need to do?
>
> Best
> Sumit

We went through something similar during security scans.  We are currently 
running Tomcat 8.5.x.  Apache httpd manages the HTTPS, so TC does not use HTTPS 
in our config.  Made 2 changes to our application's web/xml.  Maybe it will 
work in TC 9.x also?

1. Inserted "web-app_3_1.xsd" into the web-app tag schemaLocation attribute:

http://xmlns.jcp.org/xml/ns/javaee;
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
  xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd;
  version="3.1"
  metadata-complete="true">

2. Inserted cookie-config and http-only tags into the existing session-config 
tag below session-timeout:

15

true

PLEASE NOTE: I am not an expert, but hopefully this information is correct 
enough to be useful.  If not, I trust some of the real experts to correct any 
errancies.  :-) 

ALSO, it may help them to help you if you answer their questions when they ask 
you for more details.  ;-)

Don't know about the true option.

--
Cris Berneburg
CACI Lead Software Engineer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 4 Apache Events in 2019: DC Roadshow soon [etc]

2019-03-19 Thread Berneburg, Cris J. - US

-- Christopher Schultz (cs) wrote 3/12/19:
-- Rich Bowen (rb) wrote 3/6/19:

rb> * Apache Roadshow DC is in [2] weeks. Register now at 
rb> https://apachecon.com/usroadshowdc19/

cs> I'll be speaking at this event, and I'd love to meet some
cs> local Tomcat-ers. It's $25 to attend; schedule available
cs> at https://apachecon.com/usroadshowdc19/schedule.html

Thanks Chris, looking forward to it!

rb> Monday, March 25th @ George Mason University, Fairfax
cs> Hope to see some folks there,

I plan on attending.  Anyone else?

--
Cris Berneburg
CACI Lead Software Engineer

[OT?] RE: Tomcat 8.5.13 - random issue with HTTPS (blank page) - working good with HTTP

2019-03-11 Thread Berneburg, Cris J. - US

Hi Youness

Please see my comments below

-Original Message-
From: youness.dakk...@bnpparibasfortis.com 

Sent: Wednesday, March 6, 2019 7:55 AM
To: Tomcat Users List 
Subject: RE: Tomcat 8.5.13 - random issue with HTTPS (blank page) - working 
good with HTTP

> This is the content of the log4j.properties file:
>
> # Root logger
> # console will log to console (local tomcat) or stdout.log im Tomcat/logs 
> log4j.rootLogger=ALL, console
>
> # Console appender.
> log4j.appender.console=org.apache.log4j.ConsoleAppender
> log4j.appender.console.layout=org.apache.log4j.PatternLayout
> log4j.appender.console.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
>
> # File appender.
> log4j.appender.file=org.apache.log4j.RollingFileAppender
> log4j.appender.file.File=PsClient.log
> log4j.appender.file.layout=org.apache.log4j.PatternLayout
> log4j.appender.file.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
>
> # Leave an empty line at the end of the file for unix.

We use Log4j2 in our project too.  While your configuration is specified 
"log4j.properties", and ours in "log4j2.xml", I wonder if they are similar 
enough for comparison.

I see that you define 2 appenders but no logger.  The appenders define what 
Log4j2 resources are available, but did you tell it which appender to actually 
use - the logger?  We also define both console and file appenders and 
enable/disable as required with the logger.

As an analogy, you could compare defining Log4j2 resources with getting ready 
to paint.  Each resource could be likened to a paint brush.  The color and 
finish of the paint would be the layout and pattern.  Maybe you defined a thick 
brush with blue matte paint as your console appender, and a thin brush with red 
gloss paint for the file appender.  The logger keyword specifies which brush is 
in your hand.  So without using the logger keyword, neither paint brush has 
been placed into the hand for actual use.  The paint brushes (appenders) are 
just sitting there unused.

Does that make sense?

I wonder is some of the confusion with Log4j2 is the word "logger" - a generic 
term versus a keyword.

Anyway, please note that I am not an expert in Log4j2, so all this could be 
complete BS.  :-P

HTH

--
Cris Berneburg
CACI Lead Software Engineer

> -Original Message-
> From: John Dale [mailto:jcdw...@gmail.com]
> Sent: Wednesday, March 06, 2019 1:44 PM
> To: Tomcat Users List
> Subject: Re: Tomcat 8.5.13 - random issue with HTTPS (blank page) - working 
> good with HTTP
> 
> Check your log4j configuration .. make sure it's got a console appender 
> configured (based on the log file names, it would seem like a console logger 
> will be required).  If you can, try to post up your log4j configuration .. 
> there will be key classes for logging within tomcat that must be enabled.
> 
> On 3/6/19, youness.dakk...@bnpparibasfortis.com
>  wrote:
> > Those are the files on tomcat/logs/*
> >
> > - commons-daemon.-MM-DD.log
> > - tomcat.gc.log
> > - stdout.log
> > - stderr.log
> >
> > This Tomcat is used inside SAP Business Objects.
> > My questions are:
> > - How we can get better logs on Tomcat ?
> > - Do you already had that kind of behaviour --> From HTPPS you get a 
> > blank page and via HTTP it works, this is when I use the url of  
> > BIlaunchPad from SAP Business Objects
> > - Is there a tool I can use to have better logging of Tomcat ? or how 
> > to set the tomcat config to get more usefull logs
> >
> > Thanks in advance,
> > Youness

[SNIP]

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: tomcat Finding!

2018-12-26 Thread Berneburg, Cris J. - US

Hi Danyaal

dh> I'm encountering following scan finding errors
dh> and couldn't find way to mitigate this.

dh> Tomcat 8.5.32
dh> 12085
dh> Apache Tomcat Default Files
dh> The following default files were found
dh> :/nessus-check/default-404-error-page.html
dh> Delete the default index page and remove the
dh> example JSP and servlets. Follow the Tomcat
dh> or OWASP instructions to replace or modify
dh> the default error page.

We recently encountered this problem in our server scans and were able to 
mitigate the issue.

If you have not already read it, here's a Tenable forum thread about the topic. 
 While it does not provide a complete solution, it starts to explain the issue.

We started by removing the apps that came bundled in Tomcat webapps.  We 
deleted the docs, examples, and ROOT folders.

Also, we removed the  404 block from our application web.xml and 
added one to the Tomcat conf/web.xml.  Something like:


404
/NotFound.jsp


--
Cris Berneburg
CACI Lead Software Engineer
but Tomcat newbie


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [slightly OT] Re: Tomcat 9 does not work with Java 11

2018-12-12 Thread Berneburg, Cris J. - US

Hi Andi

am> Another try on a third Windows Server 2008 R2 that never contained Java or 
Tomcat.
am> I am logged in as local administrator.
am> Installed Java 11 and Tomcat 9.
am> And again same error :(
am> I would really appreciate any help.

Sorry you are going through all this trouble.  I have not tried Java 11 yet.

cs> The installer should be detecting all of that, but 
cs> at this point you are grasping at straws, anyway.

Also, as long as you're "grasping at straws" :-) I have some basic questions:

am> Since it works when I start Tomcat by startup.bat, it must be 
am> something with the service, right?

I can't remember, does your Tomcat Windows Service have a problem during 
automatic startup *and* manual startup from the Services panel (not 
startup.bat)?

Have you tried installing older versions of Java with Tomcat 9 on Windows 
Server 2008 to verify that the Tomcat Windows Service works with older Java 
versions?  Sorry, I can't remember if you tried that either.

Does Windows Event Log say anything meaningful, other than "Error Code 1" (or 
something like that)?

am> It also tried to change the START-MODE to Java.
am> Then Tomcat service started! But it could not be stopped anymore. Only 
am> by killing Java.exe.

When you run Tomcat9.exe, is the Java location specified, or do you have "use 
default" selected?  Hmm... if the Service started then that must not be the 
problem.

am> When I start Tomcat by using startup.bat it works!
[SNIP]
am> - Installed Java 11 (File: jdk-11.0.1_windows-x64_bin.exe)

Is your Windows Server 2008 32-bit or 64bit?  Hmm again... If 64-bit Java were 
installed on a 32-bit OS, Java would not run at all, so that's not it.

Guess I'm grasping at straws too.  :-)

--
Cris Berneburg
Tomcat Newbie

RE: [slightly OT] Re: Tomcat 9 does not work with Java 11

2018-12-03 Thread Berneburg, Cris J. - US

Hi Ralf

am> What I did now:
am> - removed Tomcat services by service.bat
am> - uninstalled all Tomcats (7 and 9)
am> - uninstalled all Java (was only Version 11)
am> - server reboot
am> - Installed Java 11 (File: jdk-11.0.1_windows-x64_bin.exe)
am> - reboot
am> - Installed Tomcat 9 (File: apache-tomcat-9.0.13.exe)
am> - During installation I was asked for the path to Java (field was empty). I 
entered the path to the root of Java 11: C:\Program Files\Java\jdk-11.0.1
am> - Start Service at the end of the installation
am> 
am> And again this error:
am> [2018-11-27 10:56:50] [info]  [ 3208] Commons Daemon procrun (1.1.0.0
am> 64-bit) started
am> [2018-11-27 10:56:50] [info]  [ 3208] Running 'Tomcat9' Service...
am> [2018-11-27 10:56:50] [info]  [ 3060] Starting service...
am> [2018-11-27 10:56:50] [error] [ 3060] The specified procedure could not be 
found.
am> [2018-11-27 10:56:50] [error] [ 3060] Failed creating Java
am> C:\Program Files\Java\jdk-11.0.1\bin\server\jvm.dll
am> [2018-11-27 10:56:50] [error] [ 3060] The specified procedure could not be 
found.
am> [2018-11-27 10:56:50] [error] [ 3060] ServiceStart returned 1
am> [2018-11-27 10:56:50] [error] [ 3060] The specified procedure could not be 
found.
am> [2018-11-27 10:56:50] [info]  [ 3208] Run service finished.
am> [2018-11-27 10:56:50] [info]  [ 3208] Commons Daemon procrun finished

Newbie point: My Tomcat Windows Service and Java problems were recently solved 
by running Tomcat8w.exe.  Would running Tomcat9w.exe and setting your Java 
location there be relevant to your situation?

--
Cris Berneburg
CACI Lead Software Engineer

RE: reinstall TC service after java upgrade?

2018-11-09 Thread Berneburg, Cris J. - US

Thanks André

cjb> I upgraded Java from 8u181 to 8u191 on our dev (JDK) and test 
cjb> (JRE) Windows 2012 servers today.  After doing so, the Tomcat 
cjb> 8.5(.32) Windows services would no longer start.  Removing and 
cjb> re-adding the service fixed the service.

mt> You need to reconfigure where the service looks for the JRE.
mt> Normally you'd do that with Tomcat[7|8|9]w.exe

cjb> why doesn't the service just use the JAVA_HOME or JRE_HOME
cjb> environment vars?  Is that so there can be multiple services
cjb> running with different versions of Java simultaneously?

aw> Basically yes.  Each service also picks up many other arguments there.
aw> For a complete explanation, I recommend :
aw> https://wiki.apache.org/tomcat/FAQ/Windows#Q11
aw> (disclaimer : I get bonus points each time someone reads that)

I read it, possibly again.  André ++;

Can we put a note in there saying that if a different version of Java is 
installed and the old one is removed, the registry entries are no longer valid 
and need to be refreshed by running Tomcat[#]w.exe to reference the new Java?


Note that the JVM location is also stored in the Windows Registry by 
Tomcat[#]w.exe (prunmgr) and is referenced by the service wrapper (prunsrv).  
This means that if the Java location changes, such as removing an older JVM and 
installing a newer version, the Registry entries will no longer be valid, and 
the Tomcat service will no longer run.  Running Tomcat[#]w.exe and updating the 
JVM location on the Java tab or selecting the "Use default" checkbox can remedy 
that.

"One more thing [...]"

Wait a minute.  Could it simply be that the "Use default" checkbox (for JVM) on 
the Java tab was unchecked for me?  Argh.  Cris --;

--
Cris Berneburg
CACI Lead Software Engineer

RE: reinstall TC service after java upgrade?

2018-11-07 Thread Berneburg, Cris J. - US

Thanks Mark

cjb> I upgraded Java from 8u181 to 8u191 on our dev (JDK) and test (JRE)
cjb> Windows 2012 servers today.  After doing so, the Tomcat 8.5(.32)
cjb> Windows services would no longer start.  Removing and re-adding
cjb> the service fixed the service.  I don't remember having to re-install
cjb> the TC Windows services after upgrading Java previously.  Is that normal?

mt> It is one way to fix the problem. You need to reconfigure where the
mt> service looks for the JRE. Normally you'd do that with Tomcat[7|8|9]w.exe

Wow, I actually did not know that.  I feel a bit embarrassed not knowing 
something that basic.  Did you notice my ApacheCon badge?  "Tomcat Newbie".  :-)

Then again, why doesn't the service just use the JAVA_HOME or JRE_HOME 
environment vars?  Is that so there can be multiple services running with 
different versions of Java simultaneously?

--
Cris Berneburg
CACI Lead Software Engineer

reinstall TC service after java upgrade?

2018-11-02 Thread Berneburg, Cris J. - US

Hi Folks

I upgraded Java from 8u181 to 8u191 on our dev (JDK) and test (JRE) Windows 
2012 servers today.  After doing so, the Tomcat 8.5(.32) Windows services would 
no longer start.  Removing and re-adding the service fixed the service.  I 
don't remember having to re-install the TC Windows services after upgrading 
Java previously.  Is that normal?

1. Stop TC Windows Service.
2. Uninstall old Java.
3. Install new Java.
4. Update Windows environment var - JAVA_HOME for dev, JRE_HOME for test.
5. Try to start TC Win Service - Fail.  Helpful error code of "1".  Windows 
Event Log says "incorrect function".
6. Run startup.bat manually from a command prompt - worked.
7. Run shutdown.bat to stop the manual startup.
8. Close Windows Services panel.
9. Delete TC Win Service with "Service.bat remove".
10. Recreate TC Win Service with "Service.bat install".
11. Open Windows Services panel.
12. Modify TC Win service to automatic start.
13. Start TC Win Service - OK.

Please note that I did not reboot the servers between steps.

Does the TC executable write Java-specific info to the Windows registry (or 
something) that locks the service to a specific Java location or version?  I 
don't see any registry commands in the service batch file.

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] Oracle Java 11 discussion?

2018-10-19 Thread Berneburg, Cris J. - US

Chris

cjb> large bureaucracy [...] I would not be 
cjb> surprised if there is a policy against dev kits and IDE's on 
cjb> production servers for security sake.  Tomcat (whisper: with built-in 
cjb> compiler) is approved, but is the JDK allowed?  Guess I can ask.  
cjb> Yeah, it's potentially a "distinction without a difference".

cs> Hard and fast rule: no compilers. [...]  It's a checkbox security
cs> "feature" that is all of meaningless, ineffective, and inconvenient.

Yeah, I was thinking similar things from inference.

cs> These days, most servers have all the code you'd already ever need
cs> to "compile" and run an exploit even if there were no compiler there.
cs> All you need is a nice, vulnerable pre-existing binary.

That's kinda scary.  I suppose the attitude is that as long as there are 
security updates still being published, that conforms to policy and is 
therefore OK.  Actually, what else can be done once any software has been 
released into the wild?

mt> I'd plan to stick to the LTS releases.

cjb> Meh, not my call.  Whatever the Powers That Be decide for the 
cjb> production environment, I'll probably match that in dev.

cs> They will decide to stick with Java 8, even though it's EOL. The
cs> decision will be made because (a) "there are some incompatibilities
cs> with Java 11 which are hairy to untangle" and (b) "Java 8 hasn't
cs> caused a breach, yet, so we'll probably be fine".

Interesting theory...  Care to make a friendly wager on that, say lunch and/or 
a beer?  Wait, do you have some sort of inside info?  Wager rescinded!  ;-)

My question would be how long after the 2019 EOL will Java 8 still be approved 
for use, be it official policy or unofficial inertia.  Well, at least until the 
next major vulnerability is discovered and then everyone scrambles to cover 
their behinds and upgrade Java.

cs> I'm having trouble convincing a partner vendor to move from
cs> Java *6* up to Java 8. *facepalm*

"Ha ha" (said the guy who is still in the process of upgrading from TC 6.0 to 
8.5).

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] Oracle Java 11 discussion?

2018-10-19 Thread Berneburg, Cris J. - US

Hey Chris

cjb> RAMBLE: Too bad there can't be an Apache OpenJRE umbrella project, 
cjb> with specific Apache OpenJRE [version X] sub-projects, that maintain 
cjb> JRE [version X]'s indefinitely.  One source (Apache) for all the 
cjb> different JRE's for the Java community at large, rather than depending 
cjb> on a bunch of different companies.

cs> I know it's not exactly what you meant, but...
cs> http://harmony.apache.org/
cs> You could always resurrect that project :)

Actually, that does sound like what I was thinking.  However, Harmony being 
dead since 2011 means that there hasn't been much demand for it.  I wonder if 
Oracle's new policies for Java 11 will foster a resurgence of interest in 
keeping older Java versions alive, or perhaps one version in particular...

"Java 8 Forever!"  I dunno, it kinda has the same ring to it as "Windows XP 
Forever!"

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: [OT] Oracle Java 11 discussion?

2018-10-19 Thread Berneburg, Cris J. - US

Thanks Igal

is> p.s. So happy to see that you finally moved from Tomcat 6 to 8.5.
is> Perhaps you can share that experience in a separate thread and let
is> others know if you ran into any major problems during that process.

Will do.  So far we've only run into 3 minor issues.

--
Cris Berneburg
CACI Lead Software Engineer

RE: [OT] Oracle Java 11 discussion?

2018-10-17 Thread Berneburg, Cris J. - US

Thanks Igal

mt> OpenJDK is very close to the Oracle JDK these days. I regularly run 
mt> Tomcat's unit tests with the latest OpenJDK and have yet to find an 
mt> issue that is OpenJDK specific.

is> I asked Gil Tene about this a couple of weeks ago.  Gil is a co-
is> founder of Azul Systems, an OpenJDK committer, and on the Executive
is> Committee of the JCP.  My understanding from him is that there is no
is> JDK development outside of the OpenJDK.  The Oracle developers that
is> work on the JDK commit directly to OpenJDK.  Oracle might add some
is> other things when they package their edition of the JDK for
is> distribution, but the JDK itself is the same one from OpenJDK.

Good to know.

is> The main problem with the rapid release cycle and six month support
is> is that due to late adoption, many of the bugs in a given Java
is> release are only discovered after more than six months of the release
is> date.  That means that the free support will end while bugs and
is> vulnerabilities are being discovered, forcing many organizations to
is> pay for support.

Or frequent Java installations.

RAMBLE: Too bad there can't be an Apache OpenJRE umbrella project, with 
specific Apache OpenJRE [version X] sub-projects, that maintain JRE [version 
X]'s indefinitely.  One source (Apache) for all the different JRE's for the 
Java community at large, rather than depending on a bunch of different 
companies.  The OpenJRE source code could pull from the OpenJDK repository.  A 
potential issue could be back-porting bug fixes from later versions into 
earlier ones when the source code base has shifted drastically, making merges 
difficult.

--
Cris Berneburg
CACI Lead Software Engineer

RE: [OT] Oracle Java 11 discussion?

2018-10-17 Thread Berneburg, Cris J. - US

Thanks Mark

mt> The argument for a JRE vs a JDK is that the JDK includes
mt> a compiler. The only reason Tomcat can run on a JRE and
mt> still support JSPs (which require compilation) is that
mt> Tomcat includes a Java compiler. I don't think the
mt> security argument holds much water.

I had not thought of that, and you're right (literally technically speaking).

RAMBLE: However, if I try to look at it from a point of view of a large 
bureaucracy, of which I am largely ignorant, I would not be surprised if there 
is a policy against dev kits and IDE's on production servers for security sake. 
 Tomcat (whisper: with built-in compiler) is approved, but is the JDK allowed?  
Guess I can ask.  Yeah, it's potentially a "distinction without a difference".  
Well, unless there are other tools in the JDK that can pose security risks in 
addition to the Java compiler.

mt> OpenJDK is very close to the Oracle JDK these days. I
mt> regularly run Tomcat's unit tests with the latest OpenJDK
mt> and have yet to find an issue that is OpenJDK specific.
mt>
mt> Tomcat runs happily (and is supported) on a JRE.
mt>
mt> If the JRE has passed the Java TCK then Tomcat should run
mt> on it. I don't think there is an official Tomcat position
mt> but my expectation is if a Tomcat bug (as opposed to a
mt> Java bug) appears when running on any Java implementation
mt> that has passed the TCK then the Tomcat team would treat
mt> that as a Tomcat bug and fix it.

All good to know.

cjb> I am imagining spending all my time being taken up by
cjb> Java upgrades with subsequent builds, regression testing,
cjb> red tape, and deployments

mt> I'd plan to stick to the LTS releases.

Meh, not my call.  Whatever the Powers That Be decide for the production 
environment, I'll probably match that in dev.  If they decide LT$ is the way to 
go, using the JDK will cost nothing for my dev environment anyway.  But if 
OpenJDK and frequent updates are selected ... phooey.

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[OT] Oracle Java 11 discussion?

2018-10-15 Thread Berneburg, Cris J. - US

Hi Folks

What has anyone been thinking about the upcoming Oracle Java 11 release / 
support stuff?  Frankly,  I'm confused by it all and am still trying to wrap my 
brain around it.  I have concerns about the potential implications for my 
little project, and also wonder about Tomcat at large.

No JRE - huh?  How do we run Java apps w/o a Java runtime?  Wouldn't installing 
a JDK in production be kind of a security issue?  I can imagine security 
departments not being thrilled about that.  Does Tomcat support being run on an 
OpenJRE?

Are there any implications for Tomcat?

I am imagining spending all my time being taken up by Java upgrades with 
subsequent builds, regression testing, red tape, and deployments, without 
delivering any actual new value to our customer.  :-\

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: TC 8.5 cachingAllowed=false ramifications [and potential Resource CacheSelector specification]?

2018-10-09 Thread Berneburg, Cris J. - US

Mark

cjb> SPECIFIC: The Excel files are [...] accessed only
cjb> once.  They don't need to be cached.  Is it
cjb> possible to declare only the Excel reports output
cjb> folder as non-cache-able but leave the (default)
cjb> context cache setting as-is so everything else
cjb> can be cached in the default way?  That is, set
cjb> up the Excel report output folder as a separate
cjb> "resource" with an independent cache setting?
cjb> Right now the Excel folder is embedded in the app
cjb> file system: TC/webapps/app/excel.

mt> At the moment, no. No reason why we couldn't extend
mt> the resources implementation and either add a few
mt> more options (based on path and/or filename and/or
mt> mime-type and/or whatever). Where we draw the line
mt> between 'standard' options and what requires a
mt> custom CacheSelector (ideas for better name welcome)
mt> is open to debate. Something for an enhancement request?

A bare-minimum approach that might work could be a new Resources attribute 
"cacheNotFoundResults" (default=true).

However... [LONG]

Something more robust might meet community needs better, depending on what 
folks require, rather than a one-off fix.  Need to specify what the cache 
implementation applies to.  By folder?  By file type?  What other folks want?  
I vote for an implementation by folder.

How to implement?  Move all caching specifications to a new CacheHandler class 
that the Resource references.

The 8.5 Resources docs list these attributes: allowLinking, cacheMaxSize, 
cacheObjectMaxSize, cacheTtl, cachingAllowed, className, trackLockedFiles (is 
tLF cache-related?).

The decoupled specs of Resources(a) and Cache(b) would start with:

a. Resources: allowLinking, className, trackLockedFiles, cacheMaxSize(D), 
cacheObjectMaxSize(D), cacheTtl(D), CacheSelector(new).

- (D): cacheMaxSize, cacheObjectMaxSize, cacheTtl would be deprecated but 
remain in existing TC implementations (7, 8, 9) to maintain 
backwards-compatibility.  New versions of TC (10+) would not support those 
options.

- CacheSelector would default to the default cache implementation if not 
specified.  Specifying an empty string "" would equate to "none" (no caching), 
or maybe a no-op canned class of CacheNone could be selected.

b. Cache: cacheMaxSize, cacheObjectMaxSize, cacheTtl, 
cacheNotFoundResults(new), cachedFolder(new).

- We could remove the prefix of "cache" to avoid the Smurf syndrome since it 
applies to cache anyway.
- cachingAllowed would be removed since that would be the Cache implementation 
class itself.
- cachedFolder would default to the app deployment folder.

The default cache handler CacheHandlerDefault class manages the cache for the 
app deploy folder(s) by default without changing the TC config.  You could 
specify a canned or custom cache handler at any depth for a different cache 
implementation for a specific folder set that would override the default.  That 
is, a bunch of folders would have the default cache handler by default, but a 
special (sub)folder could have a different cache implementation.

Questions / Observations:
- How to specify different cache handlers for different folders?
- What are the implications of having multiple caches?
- A cache chain or hierarchy? (override)
- Multiple CacheSelector's allowed per resource?
- One cache handler per resource?
- Nested or split Resources with one cache per sub-resource to in effect have 
multiple cache handlers?
- Cache by folder couples the TC context config to the application folder 
structure.

Meh, sounds rather complex, and my brain is tired.  :-\

--
Cris Berneburg
CACI Lead Software Engineer

RE: TC 8.5 cachingAllowed=false ramifications?

2018-10-09 Thread Berneburg, Cris J. - US

Mark

cjb> RAMBLE: The thing is, it worked in TC 6.0
cjb> but not 8.5.  Is it possible a major change
cjb> [...] Did TC 6.0 not cache files?

mt> The resources implementation was completely
mt> re-written for 8.x [...] I'm fairly sure
mt> not found results weren't cached in 6.0.x.

OK, thanks for explaining the history behind that.

cjb> GENERAL: Does the fact that a file does *not*
cjb> exist need to be cached? If a cache ping
cjb> fails, checking the file system immediately
cjb> would make new files available immediately
cjb> too, instead of after the cache expires.
cjb> (Conversely, how does it handle a file deleted
cjb> from the file system still existing in the cache?)

mt> Caching not found can improve performance.
mt> If a file is deleted, that deletion won't be
mt> detected until the associated cache entry
mt> expires.

Ha, I suspected that.  Good to know, thanks.  :-)

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: TC 8.5 cachingAllowed=false ramifications?

2018-10-09 Thread Berneburg, Cris J. - US

Thanks Chris

cjb> of TC 8.5.32 on Java 8u181, report output Excel
cjb> files won't load (immediately).  An error is
cjb> displayed to the user. [...]
cjb> 1. What are the ramifications of disabling the cache?
cjb> IOW, what are the potential side-effects? [...]
cjb> 2. Is there a "better" way to specify the setting? [...]
cjb> 3. Is there a "better" way to solve the problem? [...]

cs> Long ago, we added something similar to what you
cs> are talking about.  Basically, it was a file-
cs> upload capability for images. We waffled about
cs> whether to just map the /uploaded-images/ URL
cs> space directly to the disk and have DefaultServlet
cs> serve the bytes or to write our own servlet [...]

cs> Re-reading the documentation for 
cs> (specifically, ), it seems that:
cs>
cs>cachingAllowed="false"
cs>   base="/base/path/to/image/files/"
cs>   className="org.apache.catalina.webresources.DirResourceSet"
cs>   webAppMount="/uploaded-images/" />
cs>
cs> ... might do the trick, and it would only disable caching for that portion 
of the disk.
cs>
cs> Perhaps this would be a better solution, because
cs> it only disabled caching for a *portion* of the
cs> requests you'll be handling.

Yes, exactly!  I might experiment with something like that "next".

cjb> Is it possible to declare only the Excel reports
cjb> output folder as non-cache-able but leave the
cjb> (default) context cache setting as-is so everything
cjb> else can be cached in the default way?  That is,
cjb> set up the Excel report output folder as a separate
cjb> "resource" with an independent cache setting?
cjb> Right now the Excel folder is embedded in the app
cjb> file system: TC/webapps/app/excel.

Although I wonder if having the Excel folder embedded in the app content folder 
and specifying it in a "PostResources" clause at the same time would somehow 
conflict with the default servlet already serving it.

--
Cris Berneburg
CACI Lead Software Engineer

RE: TC 8.5 cachingAllowed=false ramifications?

Thanks Mark

cjb> Anyone have advice on, experience with, or
cjb> info about setting cachingAllowed=false?
cjb> [...]
cjb> In our testing of TC 8.5.32 on Java 8u181,
cjb> report output Excel files won't load
cjb> (immediately).  An error is displayed to
cjb> the user.  These Stack Overflow topics
cjb> below point to a cachingAllowed setting
cjb> [...]
cjb> I added 
cjb> to the  in TC/conf/context.xml,
cjb> which solved the problem.

cjb> 1. What are the ramifications of disabling
cjb> the cache?  IOW, what are the potential
cjb> side-effects?

mt> The cache keeps the contents of static files
mt> in memory to improve performance. In theory
mt> - the more of your requests that can be served
mt> from memory, the faster the response time. The
mt> side effect is a slower response time. How
mt> much actual difference this feature makes will
mt> depend on how much static content there is in
mt> your app, how frequently it is requested and
mt> how frequently it is changed.

Yeah, I was thinking something vaguely along those lines.

cjb> 2. Is there a "better" way to specify the setting?

mt> Maybe. The change you made applied that setting
mt> to ALL web applications in that Tomcat instance.
mt> If you only wanted to apply it to "/foo" then
mt> you would create:
mt> $CATALINA_BASE/conf//foo.xml
mt> [...]

OK, good to know, thanks.

cjb> 3. Is there a "better" way to solve the problem?
mt> For a given value of "better"...

:-)

mt> What is happening is that:
mt> - "something 1" requests the file
mt> - the file is not found and the cache records this
mt> - "something 2" creates the file
mt> - "something 3" requests the newly created file
mt> - the cache is still valid so the not found' response is returned
mt> - time passes, 'not found' cache response expires
mt> - "something 4" requests the newly created file which is now returned
mt> [...]
mt> What you'd need to figure out is what is "something 1"
mt> and what triggers it before "something 2". With that
mt> information, you should be able to refactor the app so
mt> "something 1" doesn't happen or happens after "something 2".

1. User client browser sents report request to TC.
2. Servlet does some stuff and calls Apache POI to generate the Excel file.
3. Servlet sends rendered JSP response, which contains HTML and Javascript.
4. Client browser processes response with Javascript, which opens a new window 
with the URL of the generated Excel file.
5. User client browser sends request for the generated Excel file from the new 
window.
6. Tomcat returns 404 not found response to new window.
7. User waits 5 to 10 seconds and clicks reload in the browser new window.
8. New client browser window sends request for the generated Excel file to TC.
9. Tomcat returns Excel file to client new window.

RAMBLE: The thing is, it worked in TC 6.0 but not 8.5.  Is it possible a major 
change in TC threading occurred, so the servlet returns the JSP response before 
the Excel file is finished being generated by POI?  No, that's not it - turning 
off caching fixes the problem.  Did TC 6.0 not cache files?

GENERAL: Does the fact that a file does *not* exist need to be cached?  If a 
cache ping fails, checking the file system immediately would make new files 
available immediately too, instead of after the cache expires.  (Conversely, 
how does it handle a file deleted from the file system still existing in the 
cache?)

SPECIFIC: The Excel files are dynamic, one-time reports, accessed only once.  
They don't need to be cached.  Is it possible to declare only the Excel reports 
output folder as non-cache-able but leave the (default) context cache setting 
as-is so everything else can be cached in the default way?  That is, set up the 
Excel report output folder as a separate "resource" with an independent cache 
setting?  Right now the Excel folder is embedded in the app file system: 
TC/webapps/app/excel.

cjb> a. This is a low-volume application.
cjb> Little traffic and few users.
cjb>
cjb> b. Seeing as we're addressing production,
cjb> we would like to implement a rapid solution.
cjb> Don't want to refactor the application,
cjb> which would take more time.

mt> Given the caveats, you solution looks to be the best (assuming performance 
is acceptable).

Thanks Mark.  It's reassuring to know the work-around is functional and not 
unreasonable.

--
Cris Berneburg
CACI Lead Software Engineer

RE: JasperException in production

Mark

cjb> getting the dreaded JasperException in production.
cjb> Don't know what changed to start causing this.  Same
cjb> thing happened in the test environment 9/4/18.  We
cjb> got around the problem in test by upgrading to Java
cjb> 8u181 and Tomcat 8.5.30.
cjb>
cjb> JRE 8u171, 32 bit
cjb> Tomcat 6.0.32, 32 bit
cjb> 
cjb> org.apache.jasper.JasperException: Unable to compile class for JSP:
cjb> An error occurred at line: 1 in the generated java file The type 
cjb> java.io.ObjectInputStream cannot be resolved.  It is indirectly 
cjb> referenced from required .class files
cjb> Stacktrace:
cjb> at org.apache.jasper.compiler.DefaultErrorHandler.javacError
cjb> (DefaultErrorHandler.java:92)
cjb> [...]

mt> The short version is that there was an upgrade to the
mt> Java version which exposed a known 'bug' in the Eclipse
mt> compiler. That 'bug' was essentially that the version
mt> of Tomcat (and hence the Eclipse compiler) was so old
mt> it was not fully compatible with Java 8.

OK, thanks for the explanation.

cjb> So our current plan is upgrade Tomcat.
mt> That should work.

Thanks for confirming.  You all have been telling me to upgrade for a while.  
:-)

cjb> It should also be possible to fix this by replacing
cjb> the ecj.jar in your existing Tomcat 6.0.x installation
cjb> with a newer version.

Good to know, just in case.  Custom tweaks get lost easily, so this will be 
"Option B".

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Apache failed to initialize connector

Hi Gael

>> >> >> On 03/10/18 12:28, Gael REYNOARD wrote:
>> >> >>> Hello everybody,
>> >> >>>
>> >> >>> OS : Windows 7 Pro x64
>> >> >>> Tomcat : 8.5.31
>> >> >>>
>> >> >>> On a test bench, I reboot Windows to test one of our C#
>> applications.
>> >> >>> Sometimes after starting the OS, my Tomcat server fails to
>> initialize
>> >> >>> because the 8080 or 8009 port would be already used.
>> >> >>
>> >> >> How are you starting Tomcat?
>> >> >>
>> >> >> Mark

gr> I disabled the automatic start of Tomcat service,
gr> it is launched a little later by my program in C #.
gr> After 314 startups of the OS, I did not have any exceptions.

gr> I did not look well enough on the internet
gr> because I found this morning a post
gr> 
(https://stackoverflow.com/questions/51666952/address-bind-exception-in-tomcat)
gr> from someone with a similar problem and Microsoft
gr> would have provided a solution since july.

I have not tried it myself, but have you considered the "Automatic (Delayed 
Start)" Startup type in your Windows service properties?  It's available on my 
TC service in Windows Server 2012 R2.

This Stack Overflow article says it waits 2 minutes:

https://stackoverflow.com/questions/11015189/automatic-vs-automatic-delayed-start/11015576#11015576

--
Cris Berneburg, Lead Software Engineer
CACI, IRMA Project
phone: 703-679-5313

-Original Message-
From: Gael REYNOARD  
Sent: Thursday, October 4, 2018 8:45 AM
To: users@tomcat.apache.org
Subject: Re: Apache failed to initialize connector

Thank you so much,

[LARGE SNIP]

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

TC 8.5 cachingAllowed=false ramifications?

Hi Folks

Anyone have advice on, experience with, or info about setting 
cachingAllowed=false?

BACKGROUND:

Our customer is suddenly getting a JasperException in production.  To solve, 
we're planning to upgrade Tomcat to 8.5.x.  In our testing of TC 8.5.32 on Java 
8u181, report output Excel files won't load (immediately).  An error is 
displayed to the user.  These Stack Overflow topics below point to a 
cachingAllowed setting:

- 
https://stackoverflow.com/questions/44852505/tomcat-8-5-takes-too-long-to-recognize-new-content

- https://stackoverflow.com/questions/3743136/how-to-disable-tomcat-caching

I added  to the  in 
TC/conf/context.xml, which solved the problem.

QUESTIONS:

1. What are the ramifications of disabling the cache?  IOW, what are the 
potential side-effects?

2. Is there a "better" way to specify the setting?

3. Is there a "better" way to solve the problem?

CAVEATS:

a. This is a low-volume application.  Little traffic and few users.

b. Seeing as we're addressing production, we would like to implement a rapid 
solution.  Don't want to refactor the application, which would take more time.

THANKS: for your time and assistance!

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

JasperException in production

OK, now we're getting the dreaded JasperException in production.  Don't know 
what changed to start causing this.  Same thing happened in the test 
environment 9/4/18.  We got around the problem in test by upgrading to Java 
8u181 and Tomcat 8.5.30.

JRE 8u171, 32 bit
Tomcat 6.0.32, 32 bit

org.apache.jasper.JasperException: Unable to compile class for JSP:
An error occurred at line: 1 in the generated java file
The type java.io.ObjectInputStream cannot be resolved.  It is indirectly 
referenced from required .class files
Stacktrace:
at 
org.apache.jasper.compiler.DefaultErrorHandler.javacError(DefaultErrorHandler.java:92)
[...]

So our current plan is upgrade Tomcat.  Another message to follow about TC 8.5 
compatibility problems, specifically cachingAllowed.

--
Cris Berneburg
CACI Lead Software Engineer

Re: help with org.apache.jasper.compiler.JDTCompiler issue?

2018-09-20 Thread Berneburg, Cris J. - US

I just remembered something.

cjb> After reverting Java and our app, the app still
cjb> won't run and still throws compilation errors.

cjb> * Staging Server - after rollback
cjb> JRE 8u171, 32 bit
cjb> Tomcat 6.0.32, 32 bit (unchanged)
cjb> App v3.3.2

cjb> * Partial stack trace:
cjb> org.apache.jasper.compiler.JDTCompiler$1 findType
cjb> SEVERE: Compilation error
cjb> org.eclipse.jdt.internal.compiler.classfmt.classFormatException
cjb> [...]

cjb> Is it possible that something on the server changed while the
cjb> older app was running, but the effects of the change were not
cjb> revealed until after the reboot?  That is, maybe everything was
cjb> resident and running in memory, but something on the disk
cjb> changed while the old version was still in use, so the old version
cjb> was broken on disk before we even started doing upgrades.  In
cjb> effect, the rug got pulled out from underneath the app, but TC
cjb> or the app didn't notice until after the new app was reloaded
cjb> into memory.  Is that possible?

We tried to do the upgrade 9/18/18 and then rolled back.

But I remember now, in my periodic testing on 9/4/18, a compile error, for 
which I decided to keep a copy of the stack trace (please see below).  So 
something *did* happen before we even started messing with the server 2 days 
ago.  But I didn't follow up on it - oops, see where that got me.

org.apache.jasper.JasperException: Unable to compile class for JSP: 

An error occurred at line: 1 in the generated java file
The type java.io.ObjectInputStream cannot be resolved. It is indirectly 
referenced from required .class files

Stacktrace:
at 
org.apache.jasper.compiler.DefaultErrorHandler.javacError(DefaultErrorHandler.java:92)
at 
org.apache.jasper.compiler.ErrorDispatcher.javacError(ErrorDispatcher.java:330)
at 
org.apache.jasper.compiler.JDTCompiler.generateClass(JDTCompiler.java:439)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:349)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:327)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:314)
at 
org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:592)
at 
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:326)
at 
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at servlets.filters.AuthFilter.doFilter(AuthFilter.java:47)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at servlets.filters.SanitizeFilter.doFilter(SanitizeFilter.java:42)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at 
org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:776)
at 
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:705)
at 
org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:898)
at 
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
at java.lang.Thread.run(Unknown Source)

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: help with org.apache.jasper.compiler.JDTCompiler issue?

2018-09-20 Thread Berneburg, Cris J. - US

Konstantin, et al

Well, it's all a moot point now.  :-)

cjb> After reverting Java and our app, the app still
cjb> won't run and still throws compilation errors.

cjb> * Staging Server - after rollback
cjb> JRE 8u171, 32 bit
cjb> Tomcat 6.0.32, 32 bit (unchanged)
cjb> App v3.3.2

kk> My guess is that the Eclipse Compiler for Java in
kk> your Tomcat 6.0.32 was released N years ago and
kk> cannot deal with Java 8u181. From the message it
kk> looks like it cannot parse some class file.

cjb> Except that we reverted both Java and our
cjb> application back to the previous versions, 8u171
cjb> and 3.3.2 respectively, and still get the error.

cjb> * Partial stack trace:
cjb> org.apache.jasper.compiler.JDTCompiler$1 findType
cjb> SEVERE: Compilation error
cjb> org.eclipse.jdt.internal.compiler.classfmt.classFormatException
cjb> [...]

kk> Option 2: Upgrade!!
kk> Tomcat 6 has reached end of life.

cjb> I knew someone would say that.  :-)  Yeah, that's "next" down the road, 
once this round of upgrades is done.

The SA installed JRE 8u181 and TC 8.5.30, which fixed the problem.  Bypassed 
CM, testing, approval process, etc. - but it works!  Still don't understand 
what went wrong.

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: help with org.apache.jasper.compiler.JDTCompiler issue?

2018-09-20 Thread Berneburg, Cris J. - US

Konstantin

Thanks for jumping in to help out.  :-)

cjb> After reverting Java and our app, the app still
cjb> won't run and still throws compilation errors.

cjb> * Staging Server - after rollback
cjb> JRE 8u171, 32 bit
cjb> Tomcat 6.0.32, 32 bit (unchanged)
cjb> App v3.3.2

kk> My guess is that the Eclipse Compiler for Java in
kk> your Tomcat 6.0.32 was released N years ago and
kk> cannot deal with Java 8u181. From the message it
kk> looks like it cannot parse some class file.

Except that we reverted both Java and our application back to the previous 
versions, 8u171 and 3.3.2 respectively, and still get the error.

cjb> * Partial stack trace:
cjb> org.apache.jasper.compiler.JDTCompiler$1 findType
cjb> SEVERE: Compilation error
cjb> org.eclipse.jdt.internal.compiler.classfmt.classFormatException
cjb> at 
org.eclipse.jdt.internal.compiler.classfmtClassFileReader.(ClassFileReader.java:342)
cjb> at org.apache.jasper.compiler.JDTCompiler$1.findType(JDTCompiler.java:206)
cjb> at org.apache.jasper.compiler.JDTCompiler$1.findType(JDTCompiler.java:163)
cjb> at 
org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(LookupEnvironment.java:96)
cjb> at 
org.eclipse.jdt.internal.compiler.lookup.UnresolvedReferenceBinding.resolve(UnresolvedReferenceBinding.java:49)
cjb> at 
org.eclipse.jdt.internal.compiler.lookup.BinaryTypeBinding.resolveType(BinaryTypeBinding.java:97)
cjb> at 
org.eclipse.jdt.internal.compiler.lookup.PackageBinding.getTypeOrPackage(PackageBinding.java:167)
cjb> at org.eclipse.jdt.internal.compiler.lookup.Scope.getType(Scope.java:2187)
cjb> at 
org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:974)
cjb> at 
org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:1164)
cjb> at 
org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.resolve(CompilationUnitDeclaration.java:366)
cjb> at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:623)
cjb> [...]

Is it possible that something on the server changed while the older app was 
running, but the effects of the change were not revealed until after the 
reboot?  That is, maybe everything was resident and running in memory, but 
something on the disk changed while the old version was still in use, so the 
old version was broken on disk before we even started doing upgrades.  In 
effect, the rug got pulled out from underneath the app, but TC or the app 
didn't notice until after the new app was reloaded into memory.  Is that 
possible?

kk> Option 2: Upgrade!!
kk> Tomcat 6 has reached end of life.

I knew someone would say that.  :-)  Yeah, that's "next" down the road, once 
this round of upgrades is done.

kk> Option 3: Switch to using a javac compiler from JDK instead of ECJ compiler.
kk> It is possible via configuration, but YMMV. It is a rarely used option.

Huh, I was wondering about the built-in compiler.  Rather than do something 
non-standard, I'd like to employ a simple solution.

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

help with org.apache.jasper.compiler.JDTCompiler issue?

2018-09-19 Thread Berneburg, Cris J. - US

Hi Folks

We can't figure out what's wrong with our staging server. After upgrading Java
and our application, Tomcat started logging "Compilation error" exceptions.
The login JSP page did not display.

After reverting Java and our app, the app still won't run and still throws
compilation errors. Tomcat is working because we can access the Manager and
Host Manager applications.

As for the app deployment, we:
1. Stop the Tomcat service.
2. Delete the contents of the tomcat folder
work/Catalina/localhost/app/org/apache/jsp.
3. Delete the contents of the app folder under webapps.
4. Copy the new app exploded structure to the webapps app folder.
5. Reboot the server (Windows Server 2012).

* Staging Server - before upgrade
JRE 8u171, 32 bit
Tomcat 6.0.32, 32 bit
App v3.3.2

* Staging Server - after upgrade
JRE 8u181, 32 bit
Tomcat 6.0.32, 32 bit (unchanged)
App v3.4.1

* Staging Server - after rollback
JRE 8u171, 32 bit
Tomcat 6.0.32, 32 bit (unchanged)
App v3.3.2

* Production Server
JRE 8u171, 32 bit
Tomcat 6.0.32, 32 bit
App v3.3.2

* Test Server
JRE 8u181, 64 bit
Tomcat 6.0.37, 64 bit
App v3.4.1
App v3.3.2

* Dev/Build Server
JDK 8u181, 64 bit
Tomcat 6.0.37, 64 bit
App v3.4.1
App v3.3.2

* Also:
a. The 32-bit staging versus 64-bit app build was not an issue in production.
b. The Tomcat revision 32 in staging versus 37 in dev/test has not been an
issue in prod.
c. This deployment method has worked for years.
d. I don't think the staging server needs either the JAVA_HOME or CATALINA_HOME
environment variables because production does not have them either.
e. The Tomcat service uses the built-in system account.

* Partial stack trace:
org.apache.jasper.compiler.JDTCompiler$1 findType
SEVERE: Compilation error
org.eclipse.jdt.internal.compiler.classfmt.classFormatException
at
org.eclipse.jdt.internal.compiler.classfmtClassFileReader.(ClassFileReader.java:342)
at org.apache.jasper.compiler.JDTCompiler$1.findType(JDTCompiler.java:206)
at org.apache.jasper.compiler.JDTCompiler$1.findType(JDTCompiler.java:163)
at
org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(LookupEnvironment.java:96)
at
org.eclipse.jdt.internal.compiler.lookup.UnresolvedReferenceBinding.resolve(UnresolvedReferenceBinding.java:49)
at
org.eclipse.jdt.internal.compiler.lookup.BinaryTypeBinding.resolveType(BinaryTypeBinding.java:97)
at
org.eclipse.jdt.internal.compiler.lookup.PackageBinding.getTypeOrPackage(PackageBinding.java:167)
at org.eclipse.jdt.internal.compiler.lookup.Scope.getType(Scope.java:2187)
at
org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:974)
at
org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:1164)
at
org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.resolve(CompilationUnitDeclaration.java:366)
at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:623)
[...]

Got any ideas? Your help would be appreciated. Thanks!

--
Cris Berneburg
CACI Lead Software Engineer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: how to prevent user access to JSP pages?

2018-08-22 Thread Berneburg, Cris J. - US

Hi Woonsan

cjb> I'd like to prevent users from requesting JSP pages directly

cjb> a. [...] adding a  for each folder.

cjb> b. [...] JSP files under the WEB-INF folder.

wk> c. Implement a servlet filter which is mapped to /* with 
wk> dispatcher options: REQUEST, INCLUDE, FORWARD. The filter may
wk> check the request URI or include/forward URI (through request
wk> attributes).

wk> The chapter 6 of the servlet spec [1] describes what Filter is,
wk> when/how it can be used, its lifecycle, etc. Dispatcher options
wk> are explained in 6.2.5.  Your servlet filter implementation may
wk> be invoked as pre-processing component before other resources
wk> or servlets.  When .jsp is accessed directly, your filter may
wk> be invoked as REQUEST dispatcher option (the default unless
wk> configured manually), you can check the resource path info
wk> through HttpRequestServlet#getRequestURI(). e.g, 
wk> /examples/hello.jsp. If you want to check the cases where the
wk> JSP is included or forwarded through RequestDispatcher, you may
wk> check servlet request attributes described in the section 9.3.1
wk> (for inclusion) or 9.4.2 (for forwarding). So, you might want to
wk> check include/forward path first and find requestURI afterward
wk> to check everything and modify the response as a result. For
wk> example, you can choose to send a 4xx response if the condition
wk> doesn't meet your requirement.  All of those are based on
wk> servlet standards.

I'm afraid this is a bit more advanced than where I currently am ATM and 
possibly what my project requires.

Also, I am reminded of the textbook phrase, "the rest is left as an exercise to 
the reader".  :-)  Or a presenter...

--
Cris Berneburg
CACI Lead Software Engineer

RE: how to prevent user access to JSP pages?

2018-08-22 Thread Berneburg, Cris J. - US

Chris

[combining messages]

cjb> Am I mistaken, but does vulnerability scanning software
cjb> seem to feed on that sort of thing?

cs> Most vulnerability scanners just try to detect your server's
cs> version and look-up any publicly-reported vulnerabilities in
cs> e.g. NVD. They are really stupid tools for the most part.

cs> If you hired a real pen tester, they would probably run one
cs> of those scanners first just to get some intel and then
cs> dive-into attacking your application e.g. with request-
cs> parameter munging.

I failed to mention that a vulnerability scanner being used actually follows 
paths in the source code.  I inferred that a clever hacker could figure out how 
to discover and exploit the vulnerabilities, that the scanner revealed, by 
reloading pages and varying parameters.

cjb> For me, it's a twofold combination of (a) security
cjb> concerns and (b) separation of responsibilities.

cjb> a. Security - shrink the attack surface.

cs> It's worth pointing-out that what's shrinking is the attack
cs> surface *of the application*, not necessarily of JSP (as a
cs> technology) itself.  While I agree 100% with (a), here, it's
cs> not because there is anything inherently risky about JSPs.
cs> It's that most people end up writing really awful JSPs that
cs> are full of holes.

Good point.  I was not faulting the JSP technology itself, but rather the 
contents of the JSP files.  While the JSP's have been refactored a few times 
already, I think the next strategic step would be to move them to a safe(r) 
place.

cjb> b. Separation of duties - I want the JSP's to simply render
cjb> pages and the non-JSP servlets to do all the heavy lifting.

cs> "separation of concerns" allows you to focus on one task in
cs> one piece of code, instead of having a JSP that needs to enforce
cs> security, sanitize inputs, query a database, manage the result
cs> set, etc. all while providing error-handling, etc. Anything that
cs> makes code more maintainable is a big +1 in my book.

+1

cs> One of the ways I have kept my code as maintainable as possible
cs> is by not using JSPs :)

OK, I'll bite.  What do you use instead of JSP?

[Chris S. replies, "Yes, folks - hook, line, AND sinker!"]  :-)

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: how to prevent user access to JSP pages?

Chris (and Mark)

Bingo!

cjb> Due to security concerns and general fussiness on my part, I'd like 
cjb> to prevent users from requesting JSP pages directly [...].  That 
cjb> way I can legitimately claim that all requests are being validated, 
cjb> input scrubbed, JSP's cannot be taken advantage of w/o their 
cjb> servlet chaperones being present, etc.

mt> I'm struggling to understand what risks exists with JSPs that don't 
mt> with Servlets. After all, a JSP is just an alternative way to write 
mt> a Servlet. Tomcat translates the .jsp file to the .java source for a 
mt> servlet, compiles it and runs it.
mt> Can you elaborate?

cs> JSP support for input validation, etc. is basically non-existent. I'm
cs> sure someone has a crappy library that can do it, and yes, you can
cs> implement everything in JSP using miles of tag libraries and stuff
cs> like that, but in the application world, that's a serious no-no.

+1

Yeah, messy.

cs> MVC (or some version of it, under various names) is the "proper" way
cs> to build software, and JSPs are relegated to the "V" portion of that
cs> paradigm.

cs> Once you have decided that JSPs are squarely in the "V" category,
cs> it's no longer appropriate for them to be treated as "C" components
cs> and therefore they should not be accessed directly.

+1

Yup, separation of responsibilities.

cs> Protecting them from direct-access is a reasonable decision for a number
cs> of reasons, including security if you have pages that cough-up sensitive
cs> information under the assumption that authentication and authorization
cs> requirements have previously been satisfied.

cs> Sure, the container's authentication and authorization should be able
cs> to protect those JSPs just fine, but the application may have other
cs> controls in place that also need to sanity-check things before the JSP
cs> takes over.

+1

Beyond merely having the bouncer allowing the person into the club, there are 
other validation and sanity-checks that need to happen, which I would prefer to 
be centralized, not in both the JSP's *and* non-JSP servlets.

cs> So, while there isn't anything particularly "dangerous" about direct-
cs> access to JSPs, there are a number of "best practices" that suggest
cs> that hiding them is a good idea.

If some authenticated user can directly access a JSP page and manipulate the 
parameters, they can keep reloading the page while varying conjured arguments 
to find and exploit potential weaknesses.  Am I mistaken, but does 
vulnerability scanning software seem to feed on that sort of thing?  Maybe it's 
just an illusion, but I feel like there is more security control if a user must 
access a servlet first.

cs> I hope that helps explain Cris's (likely) reasoning a little more.

Exact-ically.

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: how to prevent user access to JSP pages?

Hi Mark

Thanks for taking the time to reply.  :-)

cjb> Due to security concerns and general fussiness on my part, I'd like 
cjb> to prevent users from requesting JSP pages directly [...].  That 
cjb> way I can legitimately claim that all requests are being validated, 
cjb> input scrubbed, JSP's cannot be taken advantage of w/o their 
cjb> servlet chaperones being present, etc.

mt> I'm struggling to understand what risks exists with JSPs that don't
mt> with Servlets. After all, a JSP is just an alternative way to write
mt> a Servlet. Tomcat translates the .jsp file to the .java source for a
mt> servlet, compiles it and runs it.
mt> Can you elaborate?

See Chris Shultz's reply about MVC.  He pretty much nailed it.

For me, it's a twofold combination of (a) security concerns and (b) separation 
of responsibilities.

a. Security - shrink the attack surface.

b. Separation of duties - I want the JSP's to simply render pages and the 
non-JSP servlets to do all the heavy lifting.

--
Cris Berneburg, Lead Software Engineer
CACI, IRMA Project
phone: 703-679-5313


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: how to prevent user access to JSP pages?

Hi Woonsan

Thanks for providing an "option C".  :-)  There is still much for me to learn.

cjb> Due to security concerns and general fussiness on my part, I'd like 
cjb> to prevent users from requesting JSP pages directly [...].  That 
cjb> way I can legitimately claim that all requests are being validated, 
cjb> input scrubbed, JSP's cannot be taken advantage of w/o their 
cjb> servlet chaperones being present, etc.

cjb> a. [...] adding a  for each folder.

cjb> b. [...] JSP files under the WEB-INF folder.

wk> c. Implement a servlet filter which is mapped to /* with dispatcher
wk> options: REQUEST, INCLUDE, FORWARD. The filter may check the request
wk> URI or include/forward URI (through request attributes).

While I have a general idea of what you mean, I don't know how to implement 
that.  Is that a standard practice?

--
Cris Berneburg
CACI Lead Software Engineer

RE: how to prevent user access to JSP pages?

Hi Chris

Thanks for your insight and reply.

cjb> I'd like to prevent users from requesting JSP pages directly,
cjb> except for the login page.

cs> Why except for the login page? I would include the login page
cs> as something that should be fronted with a (non-JSP) servlet,
cs> even if that servlet doesn't do anything right now. It gives
cs> you great flexibility in the future.

OK, that sounds reasonable.

cjb> I want all requests to be handled by servlets.  That way I can 
cjb> legitimately claim that all requests are being validated, input 
cjb> scrubbed, JSP's cannot be taken advantage of w/o their servlet 

cs> it's easy to put a servlet in front of everything that does
cs> *not* provide everything above, but... let's just assume that's
cs> all being competently done.

Well, it is still a work in progress.

cjb> a. One way I read is by adding a  for each 
cjb> folder.  One use case is for JSP include files.  That looks
cjb> possible  but makes it seem like these are exceptions and not
cjb> the rule.  I want "deny, deny, deny" to be the default and the
cjb> one or 2 allowable JSP pages to be the exception.

cs> This is certainly doable, but it's a lot of work, and you have
cs> to maintain those blacklists as your application grows.

Agreed, and yuck.

cjb> b. Another way mentioned is by having most of the JSP files under
cjb> the WEB-INF folder.  That way the users don't have access to the
cjb> JSP's but the servlets do. [...]  Also, that would require moving
cjb> most of the JSP files.

cs> This is the way I've always seen it done, and the way I would
cs> recommend that you do it.

OK, gotcha.

cs> It *does* require that you move all your JSPs, but that's a one-time
cs> headache and it sets a precedent for the future of your project(s):
cs> put all your JSPs under /WEB-INF.
cs> You will of course also have to fix every include/forward that you
cs> have in your application

I was afraid of that.  :-/  Looks like yet another round of refactoring.  :-)

cs> fix every include/forward that you have in your application to
cs> include/forward to /WEB-INF/foo.jsp instead of just /foo.jsp.

OK, thanks for letting me know how to do that.  Will it work for both scriptlet 
<%@ include file="abc.jsp" %> and JSP  includes?

--
Cris Berneburg
CACI Lead Software Engineer

RE: how to prevent user access to JSP pages?

Hi Louis

Thanks for replying to my request for help.  :-)

cjb> Due to security concerns and general fussiness on my part, I'd like 
cjb> to prevent users from requesting JSP pages directly [...].  That 
cjb> way I can legitimately claim that all requests are being validated, 
cjb> input scrubbed, JSP's cannot be taken advantage of w/o their 
cjb> servlet chaperones being present, etc.

cjb> a. One way I read is by adding a  for each
cjb> folder. One use case is for JSP include files.  That looks possible
cjb> but makes it seem like these are exceptions and not the rule.  I
cjb> want "deny, deny, deny" to be the default and the one or 2 allowable
cjb> JSP pages to be the exception.

lz> can't you create a Security Folder and list out only the JSPs
lz> that you want to allow the users access to?  My application is
lz> a third party application so I didn't develop it but they use
lz> a folder that has a list of .jsps that I can access so I assume
lz> they have set it up in the code.

It sounds like you're suggesting something like option (a), using security 
constraints linked to folders.

lz> Or am I just telling you the end state that you want to achieve
lz> without actually coding suggesting any coding for you?

Yeah, that's an end-state, and the security folder would be one possible method 
of getting there.

--
Cris Berneburg
CACI Lead Software Engineer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: how to prevent user access to JSP pages?