JNDIRealm Authentication and Roles
Hi all,
I've been setting up user authentication based on JNDIRealm and have couple
of questions regarding the operation. I've been using one of the secured
applications that come with the examples included in Tomcat source for
testing. My setup with obfuscated names and passwords is as follows.
I have the following Realm in the default host:
ldap://ldap1.mydomain.com:389";
alternateURL="ldap://ldap2.mydomain.com:389";
connectionName="cn=connect,ou=Users,dc=mydomain,dc=com"
connectionPassword="password"
userBase="ou=Users,dc=mydomain,dc=com"
userSearch="uid={0}"
roleBase="ou=Groups,dc=mydomain,dc=com"
roleName="cn"
roleSearch="memberUid={1}"
contextFactory="org.apache.catalina.ldap.realm.LdapTlsContextFactory"/>
...
and have modified the security constraint roles in the web.xml of the
examples application to match my LDAP groups:
MyCompany Users
...
tomcat-users
MyCompany Users
Now when I hit the protected application,
https://myserver/examples/jsp/security/protected/, I can successfully login
but only if the role-name is set to "MyCompany Users". When I replace it
with the tomcat-users, comment it out and uncomment the tomcat-users role
name, the authentication fails. The following are the traces from the
Tomcat log and LDAP log:
Oct 07, 2014 2:35:06 PM org.apache.catalina.realm.RealmBase hasRole
FINE: Username user1 does NOT have role tomcat-users
Oct 07, 2014 2:35:06 PM org.apache.catalina.realm.RealmBase
hasResourcePermission
FINE: No role found: tomcat-users
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=1 BIND
dn="cn=connect,ou=Users,dc=mydomain,dc=com" method=128
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=1 BIND
dn="cn=connect,ou=Users,dc=mydomain,dc=com" mech=SIMPLE ssf=0
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=1 RESULT tag=97 err=0 text=
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=2 SRCH
base="ou=Users,dc=mydomain,dc=com" scope=1 deref=3 filter="(uid=user1)"
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=2 SRCH attr=1.1
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=2 SEARCH RESULT tag=101
err=0 nentries=1 text=
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=3 BIND anonymous
mech=implicit ssf=0
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=3 BIND
dn="uid=user1,ou=Users,dc=mydomain,dc=com" method=128
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=3 BIND
dn="uid=user1,ou=Users,dc=mydomain,dc=com" mech=SIMPLE ssf=0
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=3 RESULT tag=97 err=0 text=
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=4 SRCH base="" scope=0
deref=3 filter="(objectClass=*)"
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=4 SEARCH RESULT tag=101
err=0 nentries=1 text=
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=5 BIND anonymous
mech=implicit ssf=0
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=5 BIND
dn="cn=connect,ou=Users,dc=mydomain,dc=com" method=128
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=5 BIND
dn="cn=connect,ou=Users,dc=mydomain,dc=com" mech=SIMPLE ssf=0
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=5 RESULT tag=97 err=0 text=
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=6 SRCH
base="ou=Groups,dc=mydomain,dc=com" scope=1 deref=3
filter="(memberUid=user1)"
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=6 SRCH attr=cn
Oct 7 14:35:06 ldap1 slapd[1367]: conn=1123 op=6 SEARCH RESULT tag=101
err=0 nentries=2 text=
So the LDAP server returns 2 entries for the role query (filtering by
attr=cn) which can be confirmed by the following LDAP command doing the
same:
$ ldapsearch -LLL -Z -H ldap://myldap:389/ -D
"cn=connect,ou=Users,dc=mydomain,dc=com" -W -b ou=Groups,dc=mydomain,dc=com
"(memberUid=user1)" cn
Enter LDAP Password:
dn: cn=tomcat-users,ou=Groups,dc=mydomain,dc=com
cn: MyCompany Users
cn: tomcat-users
dn: cn=user1,ou=Groups,dc=mydomain,dc=com
cn: MyCompany Users
cn: user1
Not sure if understand it correctly, but I thought the Realm would loop
through the cn's returned and find the right one before it fails but looks
like it picks up the first cn only? Is there something I can modify in my
Realm without changing anything on the LDAP side to fix this?
Thanks,
Igor
Re: Connection count explosion due to thread http-nio-80-ClientPoller-x death
On 06/10/2014 10:11, Lars Engholm Johansen wrote: > Hi all, > > I have good news as I have identified the reason for the devastating > NioEndpoint.Poller thread death: > > In rare circumstances a ConcurrentModification can occur in the Poller's > connection timeout handling called from OUTSIDE the try-catch(Throwable) of > Poller.run() > > java.util.ConcurrentModificationException > at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922) > at java.util.HashMap$KeyIterator.next(HashMap.java:956) > at > java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067) > at > org.apache.tomcat.util.net.NioEndpoint$Poller.timeout(NioEndpoint.java:1437) > at > org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1143) > at java.lang.Thread.run(Thread.java:745) > > Somehow the Poller's Selector object gets modified from another thread. Any idea how? I've been looking through that code for some time now (this stack trace appears to be from 7.0.55 for those that want to look at this themselves) and I can't see anywhere where the selector's keyset is accessed by more than one thread. > As a remedy until fixed properly by the Tomcat team, I have added a > try-catch(ConcurrentModificationException) surrounding the for loop in > Poller.timeout(). > That way, in case of the rare problem, a full iteration of the Selector > will be retried in the next call to Poller.timeout(). That seems like a reasonable work-around but before we start making changes to the Tomcat code I'd really like to understand the root cause(s) of the issue else we might not be fixing the actual issue and could make it worse for some folks. Mark > > I am really happy now as all our production servers have been rock stable > for two weeks now. > > Best regards to all, > Lars Engholm Johansen > > > On Thu, Sep 18, 2014 at 7:03 PM, Filip Hanik wrote: > >> Thanks Lars, if you are indeed experiencing a non caught error, let us know >> what it is. >> >> On Thu, Sep 18, 2014 at 2:30 AM, Lars Engholm Johansen >> wrote: >> >>> Thanks guys for all the feedback. >>> >>> I have tried the following suggested tasks: >>> >>>- Upgrading Tomcat to the newest 7.0.55 on all our servers -> Problem >>>still persists >>>- Force a System.gc() when connection count is on the loose -> >>>Connection count is not dropping >>>- Lowering the log level of NioEndpoint class that contains the Poller >>>code -> No info about why the poller thread exits in any tomcat logs >>>- Reverting the JVM stack size per thread to the default is discussed >>>previously -> Problem still persists >>> >>> I have now checked out the NioEndpoint source code and recompiled it >> with a >>> logging try-catch surrounding the whole of the Poller.run() >> implementation >>> as I noticed that the outer try-catch here only catches OOME. >>> I will report back with my findings as soon as the problem arises again. >>> >>> /Lars >>> >>> >>> >>> On Fri, Jun 27, 2014 at 9:02 PM, Christopher Schultz < >>> ch...@christopherschultz.net> wrote: >>> > Filip, > > On 6/27/14, 11:36 AM, Filip Hanik wrote: >> Are there any log entries that would indicate that the poller >> thread has died? This/these thread/s start when Tomcat starts. and >> a stack over flow on a processing thread should never affect the >> poller thread. > > OP reported in the initial post that the thread had disappeared: > > On 6/16/14, 5:40 AM, Lars Engholm Johansen wrote: >> We have no output in tomcat or our logs at the time when this event >> occurs. The only sign is when comparing full java thread dump with >> a dump from a newly launched Tomcat: >> >> One of http-nio-80-ClientPoller-0 or http-nio-80-ClientPoller-1 >> is missing/has died. > > -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org >>> >> > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Maven Tomcat 6/7/8 plugin
Perfect! I didn't know about this one. Thanks! 2014-10-06 13:37 GMT+02:00 Daniel Mikusa : > On Sun, Oct 5, 2014 at 2:00 PM, Matthias Hryniszak > wrote: > >> It seems I might have expressed myself purely. Let me rephrase: >> >> I'm looking for RemoteIpValve support in maven-tomcat7-plugin. > > > Perhaps you could use RemoteIPFilter instead? You can configure that > through web.xml. > > > http://tomcat.apache.org/tomcat-7.0-doc/config/filter.html#Remote_IP_Filter > > Dan > > >> I know the >> valve itself exists since long before version 7 and I am already using it >> in production. What I'm on to here is running Maven-managed web >> applications in development and as far as I can see it's been added ~2 >> weeks after the release of 2.2 version of that plugin >> >> Alternatively if you know of a way to persuade Apache CXF to present the >> endpoint URL using HTTPS scheme when running mvn tomcat7:run. CXF does it >> automatically if the RemoteIpValve is installed and proper header exists >> (as in the isSecure() method returns true). Otherwise it does just HTTP >> and >> that in turn makes my local HAProxy do a 302 to HTTPS which ends up in an >> infinite loop and that's not exactly the result I was hoping for... >> >> Cheers, >> Matthias >> >> 2014-10-05 19:31 GMT+02:00 Konstantin Kolinko : >> >> > 2014-10-05 18:01 GMT+04:00 Matthias Hryniszak : >> > > Hi all, >> > > >> > > I'm new to this list so let me briefly introduce myself. My name is >> > > Matthias and I live in Poland. I'm a software architect for Lumesse, >> an >> > > British software development company. The core of our development is >> > talent >> > > acquisition and talent management software. The part I'm working with >> is >> > > e-learning solutions. >> > > >> > > Now that you all know me let me ask you a question: are there any >> plans >> > on >> > > releasing a new version of the maven-tomcat7-plugin and/or an upgrade >> to >> > > maven-tomcat8-plugin? We're falling short a bit on the lack of remote >> IP >> > > valve in the current release >> > >> > RemoteIpValve does exist in Tomcat 7 >> > >> http://tomcat.apache.org/tomcat-7.0-doc/config/valve.html#Remote_IP_Valve >> > >> > > and I was wondering if that's going to be >> > > released any time soon. Please note the last release (2.2) was >> November >> > > 2013 and there's been lots of changes so it'd be great to have them >> > > stabilized and released - not to mention the upgrade over embedded >> Tomcat >> > > version. >> > >> > It is being developed on a branch, >> > http://svn.apache.org/viewvc/tomcat/maven-plugin/branches/tc8.x/ >> > >> > See archives of tomcat dev mailing list for discussions. >> > >> > Thus far, integration tests are failing, >> > http://markmail.org/message/xc5r6yycrvmjh2vx >> > >> > >> > Best regards, >> > Konstantin Kolinko >> > >> > - >> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> > For additional commands, e-mail: users-h...@tomcat.apache.org >> > >> > >> > >
Re: Maven Tomcat 6/7/8 plugin
On Sun, Oct 5, 2014 at 2:00 PM, Matthias Hryniszak wrote: > It seems I might have expressed myself purely. Let me rephrase: > > I'm looking for RemoteIpValve support in maven-tomcat7-plugin. Perhaps you could use RemoteIPFilter instead? You can configure that through web.xml. http://tomcat.apache.org/tomcat-7.0-doc/config/filter.html#Remote_IP_Filter Dan > I know the > valve itself exists since long before version 7 and I am already using it > in production. What I'm on to here is running Maven-managed web > applications in development and as far as I can see it's been added ~2 > weeks after the release of 2.2 version of that plugin > > Alternatively if you know of a way to persuade Apache CXF to present the > endpoint URL using HTTPS scheme when running mvn tomcat7:run. CXF does it > automatically if the RemoteIpValve is installed and proper header exists > (as in the isSecure() method returns true). Otherwise it does just HTTP and > that in turn makes my local HAProxy do a 302 to HTTPS which ends up in an > infinite loop and that's not exactly the result I was hoping for... > > Cheers, > Matthias > > 2014-10-05 19:31 GMT+02:00 Konstantin Kolinko : > > > 2014-10-05 18:01 GMT+04:00 Matthias Hryniszak : > > > Hi all, > > > > > > I'm new to this list so let me briefly introduce myself. My name is > > > Matthias and I live in Poland. I'm a software architect for Lumesse, an > > > British software development company. The core of our development is > > talent > > > acquisition and talent management software. The part I'm working with > is > > > e-learning solutions. > > > > > > Now that you all know me let me ask you a question: are there any plans > > on > > > releasing a new version of the maven-tomcat7-plugin and/or an upgrade > to > > > maven-tomcat8-plugin? We're falling short a bit on the lack of remote > IP > > > valve in the current release > > > > RemoteIpValve does exist in Tomcat 7 > > > http://tomcat.apache.org/tomcat-7.0-doc/config/valve.html#Remote_IP_Valve > > > > > and I was wondering if that's going to be > > > released any time soon. Please note the last release (2.2) was November > > > 2013 and there's been lots of changes so it'd be great to have them > > > stabilized and released - not to mention the upgrade over embedded > Tomcat > > > version. > > > > It is being developed on a branch, > > http://svn.apache.org/viewvc/tomcat/maven-plugin/branches/tc8.x/ > > > > See archives of tomcat dev mailing list for discussions. > > > > Thus far, integration tests are failing, > > http://markmail.org/message/xc5r6yycrvmjh2vx > > > > > > Best regards, > > Konstantin Kolinko > > > > - > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > >
RE: tomcat crash problem (INTERNAL)
Thanks Ognjen. Regards, Subbu. -Original Message- From: Ognjen Blagojevic [mailto:ognjen.d.blagoje...@gmail.com] Sent: 6. oktober 2014 11:48 To: Tomcat Users List Subject: Re: tomcat crash problem (INTERNAL) Subbu, On 6.10.2014 10:27, bala-subrahmanyam.bha...@telenor.com wrote: > Could you please suggest one best open source java profiler for analysing and > monitoring the tomcat server. I don't know which (open source) one is the best, but you may try your luck with: 1. MAT (http://www.eclipse.org/mat/), which analyzes heap dumps. Just dump the server memory while the server is running and later open dump files with MAT. 2. VisualVM can monitor your server memory usage, measure surviving generations, and compare memory snapshots -- so you could detect what is causing memory leak. -Ognjen - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat crash problem (INTERNAL)
Subbu, On 6.10.2014 10:27, bala-subrahmanyam.bha...@telenor.com wrote: Could you please suggest one best open source java profiler for analysing and monitoring the tomcat server. I don't know which (open source) one is the best, but you may try your luck with: 1. MAT (http://www.eclipse.org/mat/), which analyzes heap dumps. Just dump the server memory while the server is running and later open dump files with MAT. 2. VisualVM can monitor your server memory usage, measure surviving generations, and compare memory snapshots -- so you could detect what is causing memory leak. -Ognjen - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Connection count explosion due to thread http-nio-80-ClientPoller-x death
Hi all, I have good news as I have identified the reason for the devastating NioEndpoint.Poller thread death: In rare circumstances a ConcurrentModification can occur in the Poller's connection timeout handling called from OUTSIDE the try-catch(Throwable) of Poller.run() java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922) at java.util.HashMap$KeyIterator.next(HashMap.java:956) at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067) at org.apache.tomcat.util.net.NioEndpoint$Poller.timeout(NioEndpoint.java:1437) at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1143) at java.lang.Thread.run(Thread.java:745) Somehow the Poller's Selector object gets modified from another thread. As a remedy until fixed properly by the Tomcat team, I have added a try-catch(ConcurrentModificationException) surrounding the for loop in Poller.timeout(). That way, in case of the rare problem, a full iteration of the Selector will be retried in the next call to Poller.timeout(). I am really happy now as all our production servers have been rock stable for two weeks now. Best regards to all, Lars Engholm Johansen On Thu, Sep 18, 2014 at 7:03 PM, Filip Hanik wrote: > Thanks Lars, if you are indeed experiencing a non caught error, let us know > what it is. > > On Thu, Sep 18, 2014 at 2:30 AM, Lars Engholm Johansen > wrote: > > > Thanks guys for all the feedback. > > > > I have tried the following suggested tasks: > > > >- Upgrading Tomcat to the newest 7.0.55 on all our servers -> Problem > >still persists > >- Force a System.gc() when connection count is on the loose -> > >Connection count is not dropping > >- Lowering the log level of NioEndpoint class that contains the Poller > >code -> No info about why the poller thread exits in any tomcat logs > >- Reverting the JVM stack size per thread to the default is discussed > >previously -> Problem still persists > > > > I have now checked out the NioEndpoint source code and recompiled it > with a > > logging try-catch surrounding the whole of the Poller.run() > implementation > > as I noticed that the outer try-catch here only catches OOME. > > I will report back with my findings as soon as the problem arises again. > > > > /Lars > > > > > > > > On Fri, Jun 27, 2014 at 9:02 PM, Christopher Schultz < > > ch...@christopherschultz.net> wrote: > > > > > -BEGIN PGP SIGNED MESSAGE- > > > Hash: SHA256 > > > > > > Filip, > > > > > > On 6/27/14, 11:36 AM, Filip Hanik wrote: > > > > Are there any log entries that would indicate that the poller > > > > thread has died? This/these thread/s start when Tomcat starts. and > > > > a stack over flow on a processing thread should never affect the > > > > poller thread. > > > > > > OP reported in the initial post that the thread had disappeared: > > > > > > On 6/16/14, 5:40 AM, Lars Engholm Johansen wrote: > > > > We have no output in tomcat or our logs at the time when this event > > > > occurs. The only sign is when comparing full java thread dump with > > > > a dump from a newly launched Tomcat: > > > > > > > > One of http-nio-80-ClientPoller-0 or http-nio-80-ClientPoller-1 > > > > is missing/has died. > > > > > > - -chris > > > -BEGIN PGP SIGNATURE- > > > Version: GnuPG v1 > > > Comment: GPGTools - http://gpgtools.org > > > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > > > > > iQIcBAEBCAAGBQJTrb+yAAoJEBzwKT+lPKRYhYEP/05kiei/EUFhtxL6RMIl70Ok > > > cb3I9XEvrQDBTkEDnGLvxw8MQSs6ocHaxdEOxzie289sYxvkuLWxOsKpikWkuUHH > > > pEgHM5WuGuCS2AmcrTGiH6WPCnNAj8YM/zyx25NZOn8turWIbvh8GRzBFf265qP5 > > > 79z2Vb15NisYyNEqvkWHvli5CeDeOW2fgHcgv5Ec5fWb1/KyXAyVtRmEWnHpy/LB > > > j/VLjzbBtFSJGT64W4i572qQ7C+f/XRgNzV6Fh/53gwPf+ggz5vKS9XEQEpa5SOz > > > rlTrWuVs+WehBoCLE9TZB2J+argV7noqSQDumYcXeSf/4THkfhbhAlcBKXa/YLgH > > > Paip710VV6S+9K1dAZOt4i1h28YXZ+qNviO6b/auo1DEdt21ezpklEOQyZbQcHYf > > > H4VZ2mcSaMQo3QpWpze6QxvSsRZFAofpkLoqCRfsORlnV2c2xfjhRC1YtZ0sshfM > > > zNnWQCEjRe5V+UB69mtjatJrDG16qjTcUZQlot3r4zxdjMq5D0W9XmC6WH2eCXhl > > > aeH8SMISdn4GcYGMoUm7hWSWHs5azyBPma9AWJfYC+mLk8UbmvLP9gZN+KWenWOr > > > xLiqCgMUvpLiOFsbNs8oWMDWGW59xT2zBjS3Aa20ZYJP/GeLWJkOrAPwTeqIaXG+ > > > tV1WjkDkejPrC4WWKwzm > > > =sTia > > > -END PGP SIGNATURE- > > > > > > - > > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > > > > > >
Re: SecureRandom instance for session ID generation using [SHA1PRNG] took [510,962] milliseconds !
Le 03/10/2014 20:41, Rainer Jung a écrit : Am 03.10.2014 um 14:01 schrieb Christopher Schultz: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Martin, On 10/3/14 5:48 AM, Martin Hamant wrote: Le 03/10/2014 11:26, Martin Hamant a écrit : The virtual (qemu) server runs with 4GB RAM Sorry, The hypervisor is KVM. The VM is running on top of OpenStack So... This could lead somewhere as I am reading http://blog.dustinkirkland.com/2012/10/entropy-or-lack-thereof-in-openstack.html OpenStack or not, running on a VM usually means that the underlying OS is providing the source of entropy. If your physical machine is heavily virtualized, you may have multiple entropy sinks constantly draining your source(s() of entropy. If you wait for a while, things will recover. If you find you are constantly blocking waiting for more randomness to be available from your random source, you basically have 3 options: 1. Suffer through it. Just keep waiting. 2. Use a poor source of randomness, like /dev/urandom on Linux. I wouldn't recommend this for any kind of production deployment, since the entropy source is "watered-down". You can't rely on it for important things like encryption (including SSL) and really anything that requires random numbers that are as random as possible (like session ids). 3. Get yourself a hardware entropy source. You can buy USB keys that do this kind of thing. Make sure whatever you get is compatible with your OS and accessible by Java (better yet, get one that will simply dump its randomness into /dev/random). ... and in case you are heading for the urandom solution and are sing JDK before 8, you should use e.g. Thanks both of you for your help. -Djava.security.egd=file:/dev//urandom and *not* -Djava.security.egd=file:/dev/urandom And what about using haveged (so no need to alter setenv.sh) in the VM VS using /dev/urandom ? I read about it here http://security.stackexchange.com/questions/34523/is-it-appropriate-to-use-haveged-as-a-source-of-entropy-on-virtual-machines The small C program returns values between 20-30 in my VM, but as specified it doesn't guarantee anything... - Waiting 10min for a tomcat to start is a pain - getting a USB hardware device for that is like walking on head. So... I consider using haveged or urandom - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: tomcat crash problem (INTERNAL)
Hi Ognjen, Could you please suggest one best open source java profiler for analysing and monitoring the tomcat server. Regards, Subbu. -Original Message- From: Ognjen Blagojevic [mailto:ognjen.d.blagoje...@gmail.com] Sent: 3. oktober 2014 12:47 To: Tomcat Users List Subject: Re: tomcat crash problem (INTERNAL) Subbu, On 3.10.2014 10:25, bala-subrahmanyam.bha...@telenor.com wrote: > Hi Ognjen, > > Tomcat is crashing with the below error message. > > java.lang.OutOfMemoryError: GC overhead limit exceeded Please, reply below the quotes, it is standard on this list. Tomcat have small memory footprint, way below 3 GB you allocate for JVM. So, it is probably your application data filling this 3 GB of memory. The above error says that garbage collector is struggling to release the memory, but it consumes to much CPU cycles and releases too little memory. This usually results in dramatic server slowdowns, so JVM decides to throw an error and inform you about the problem. You should analyze your application heap usage with the profiler to determine what objects are holding references so GC is unable to free more memory. If you don't need those objects, clear the references. If you really need all those objects, throw more memory at the server. Please read: http://wiki.apache.org/tomcat/OutOfMemory -Ognjen - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: a problem: tomcat exits unexpectedly
bo zhao wrote:
Warnier,
Thank you for your reply.
AbstractScheduleTaskProcess class use the ExecutorService to create
thread and runs some custom tasks. These tasks will be done in a few
seconds. Sample code is following:
for (final List list : lists) {
executor.submit(new Callable() {
@Override
public Object call() throws Exception {
try {
executeTasks(list);
..
But I don't think that this class causes the serious prolbem. the max
size of "lists" is 4 -- that is a parameter of application.
In our server, we have 12 tomcat instances that run the different
application, but the default value of os parameter(open files and max
processes) is 1024, at first, I modified these value to 65535. but some of
these instance had still exited unexpectedly, I has found the same things
in these log: pause, reinit, stop. So I think that the os resource limit
maybe cause all of these, especially the parameter of the "open files".
I don't know well the tomcat, so I maybe has not found the key cause
and I expect your reply.
In such circumstances, you can (probably) consider Tomcat as "just a process" like any
other. It looks like something in that process is holding on to a lot of open files/sockets.
The Linux utility "lsof" provides (among a zillion other things) a good way to list all
the files/sockets linked to one process.
Try it for one of your Tomcat instances, and probably the names of these files/sockets
will give you a hint.
Thank you very much.
On Sun, Sep 28, 2014 at 6:37 PM, André Warnier wrote:
bo zhao wrote:
I had modified the os ulimit parameter: open files and max processes,
after
that, Tomcat does't exit unexpectedly, and the log of Tomcat have not the
"... pause, ... init". It seems work normally.
But I still don't know the reason. I want to know if the two parameter is
the key to solve probelm and how the two parameter of os limit cause the
exit of tomcat?
I believe that you are understanding this wrongly. These paraneters are
not the /cause/ of the problem, they are a /symptom/. The default value fo
these parameters is set so that they are ok for any normal process/system,
and rarely need to be adjusted.
By increasing their value, you are moving the problem further down the
line, but are not solving the real (possible) problem.
Ognjen is asking a question below. Why do you not start by answering it ?
On Fri, Sep 19, 2014 at 5:17 PM, Ognjen Blagojevic <
ognjen.d.blagoje...@gmail.com> wrote:
Zhao,
On 19.9.2014 3:42, bo zhao wrote:
but I can't find any error message in the log? what causes the tomcat to
pause and stop?
One of the suspects for restarts and shutdown seems to be the class
com.jd.clover.center.service.AbstractScheduleTaskProcess, as there is a
log message regarding it, at the first line of each log excerpt you sent.
What is AbstractScheduleTaskProcess class for?
-Ognjen
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: a problem: tomcat exits unexpectedly
Warnier,
Thank you for your reply.
AbstractScheduleTaskProcess class use the ExecutorService to create
thread and runs some custom tasks. These tasks will be done in a few
seconds. Sample code is following:
for (final List list : lists) {
executor.submit(new Callable() {
@Override
public Object call() throws Exception {
try {
executeTasks(list);
..
But I don't think that this class causes the serious prolbem. the max
size of "lists" is 4 -- that is a parameter of application.
In our server, we have 12 tomcat instances that run the different
application, but the default value of os parameter(open files and max
processes) is 1024, at first, I modified these value to 65535. but some of
these instance had still exited unexpectedly, I has found the same things
in these log: pause, reinit, stop. So I think that the os resource limit
maybe cause all of these, especially the parameter of the "open files".
I don't know well the tomcat, so I maybe has not found the key cause
and I expect your reply.
Thank you very much.
On Sun, Sep 28, 2014 at 6:37 PM, André Warnier wrote:
> bo zhao wrote:
>
>> I had modified the os ulimit parameter: open files and max processes,
>> after
>> that, Tomcat does't exit unexpectedly, and the log of Tomcat have not the
>> "... pause, ... init". It seems work normally.
>>
>> But I still don't know the reason. I want to know if the two parameter is
>> the key to solve probelm and how the two parameter of os limit cause the
>> exit of tomcat?
>>
>
> I believe that you are understanding this wrongly. These paraneters are
> not the /cause/ of the problem, they are a /symptom/. The default value fo
> these parameters is set so that they are ok for any normal process/system,
> and rarely need to be adjusted.
> By increasing their value, you are moving the problem further down the
> line, but are not solving the real (possible) problem.
>
> Ognjen is asking a question below. Why do you not start by answering it ?
>
>
>
>
>> On Fri, Sep 19, 2014 at 5:17 PM, Ognjen Blagojevic <
>> ognjen.d.blagoje...@gmail.com> wrote:
>>
>> Zhao,
>>>
>>> On 19.9.2014 3:42, bo zhao wrote:
>>>
>>> but I can't find any error message in the log? what causes the tomcat to
pause and stop?
One of the suspects for restarts and shutdown seems to be the class
>>> com.jd.clover.center.service.AbstractScheduleTaskProcess, as there is a
>>> log message regarding it, at the first line of each log excerpt you sent.
>>>
>>> What is AbstractScheduleTaskProcess class for?
>>>
>>> -Ognjen
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>
>>>
>>>
>>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
