Re: Fwd: Reverse proxy and SSL redirect
On Wed, Jul 1, 2020 at 3:26 AM Mark Thomas wrote: > > On 01/07/2020 00:41, rugman66 . wrote: > > On Wed, Apr 22, 2020 at 9:21 AM Mark Thomas wrote: > >> > >> On 22/04/2020 00:11, rugman66 . wrote: > >> > >> > >> > >>>Tomcat log (I'm trying to get more debug level logging) > >>> 2020-04-21 13:39:33 INFO app.CompletionRestController > >>> Unsupported Media Type in Header > >>> > >>> Postman > >>>415 Unsupported Media Type > >>> > >>> GET URL > >>> http://server.com/app/api/completions.json?username=foo > >>> > >>> Both Tomcat and Apache are running SSL because all internal endpoints > >>> are required to be secure. > >> > >> Looks like the app is generating the error. That moves us forwards. > >> > >> Try enabling the RequestDumperFilter. That should dump the full set of > >> request headers received which will hopefully help explain what is going on. > >> > >> Mark > > > > Hi Mark, > > > > Was on unplanned leave for the past few months, but back. > > > > I did try to enable RequestDumperFilter, however the file was created > > but no log entries created. I did find something interesting. When I > > test in Postman with > > HTTP it does redirect to HTTPD but throws the error. However when I > > change the URL in Postman using HTTPD I get the expected reply and see > > the > > proxy is indeed working. It's only throwing the error when the > > redirect occurs. Seems to me the issue lies there, but I still can't > > find a resolution. Any > > suggestions would be appreciated. > > You need to find a way to see the full traffic for both client<->httpd > and httpd<->Tomcat. > > Wireshark is one option. You'll need to configure it to decrypt the TLS. > > The access logs will also confirm whether requests are passed to Tomcat > or handled by httpd. > > Mark Unfortunately I cannot use wireshark as this is in one of our data centers, and information security would flag packet sniffing as malicious. However I did record the Apache access log entry for one attempt and Apache error log entries from three separate attempts. Interestingly enough all three differ in length. Also included the catalina.out log entry. Below are the log snipents. Appreciate your time -John *Tomcat* catalina.out: 2020-07-01 13:18:59 INFO app.CompletionRestController Unsupported Media Type in Header *Apache* access log: 10.24.36.111 - - [01/Jul/2020:13:18:59 -0700] "GET /app/api/completions.json?username=me HTTP/1.1" 415 46 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0" error log: [Wed Jul 01 10:42:24.994833 2020] [ssl:info] [pid 4874] [client 10.24.36.111:54100] AH01964: Connection to child 2 established (server englearn-app3.foo.com:443) [Wed Jul 01 10:42:25.011695 2020] [proxy:debug] [pid 72913] proxy_util.c(1843): AH00925: initializing worker proxy:reverse shared [Wed Jul 01 10:42:25.011740 2020] [proxy:debug] [pid 72913] proxy_util.c(1885): AH00927: initializing worker proxy:reverse local [Wed Jul 01 10:42:25.011903 2020] [proxy:debug] [pid 72913] proxy_util.c(1936): AH00931: initialized single connection worker in child 72913 for (*) [Wed Jul 01 10:42:25.011912 2020] [proxy:debug] [pid 72913] proxy_util.c(1843): AH00925: initializing worker https://englearn-app3.foo.com:8443/app shared [Wed Jul 01 10:42:25.011917 2020] [proxy:debug] [pid 72913] proxy_util.c(1885): AH00927: initializing worker https://englearn-app3.foo.com:8443/app local [Wed Jul 01 10:42:25.011934 2020] [proxy:debug] [pid 72913] proxy_util.c(1936): AH00931: initialized single connection worker in child 72913 for (englearn-app3.foo.com) [Wed Jul 01 10:42:25.041766 2020] [proxy:trace2] [pid 4874] proxy_util.c(1985): [client 10.24.36.111:54100] https: found worker https://englearn-app3.foo.com:8443/app for https://englearn-app3.foo.com:8443/app/api/completions.json?username=me, referer: http://englearn-app3.foo.com/app/api/completions.json?username=me [Wed Jul 01 10:42:25.041787 2020] [proxy:debug] [pid 4874] mod_proxy.c(1123): [client 10.24.36.111:54100] AH01143: Running scheme https handler (attempt 0), referer: http://englearn-app3.foo.com/app/api/completions.json?username=me [Wed Jul 01 10:42:25.041804 2020] [proxy:debug] [pid 4874] proxy_util.c(2203): AH00942: HTTPS: has acquired connection for ( englearn-app3.foo.com) [Wed Jul 01 10:42:25.041826 2020] [proxy:debug] [pid 4874] proxy_util.c(2256): [client 10.24.36.111:54100] AH00944: connecting https://englearn-app3.foo.com:8443/app/api/completions.json?username=me to englearn-app3.foo.com:8443, referer: http://englearn-app3.foo.com/app/api/completions.json?username=me [Wed Jul 01 10:42:25.042535 2020] [proxy:debug] [pid 4874] proxy_util.c(2426): [client 10.24.36.111:54100] AH00947: connected /app/api/completions.json?username=me to englearn-app3.foo.com:8443, referer: http://englearn-app3.foo.com/app/api/completions.json?username=me [Wed Jul 01 10:42:25.042561 2020] [proxy:trace2] [pid 4874] proxy_util.c(2768): HTTPS:
Re: Having trouble with tomcat 7 installation on RHEL 7.8 power pc
On Wed, Jul 1, 2020 at 5:24 PM calder wrote: > On Wed, Jul 1, 2020, 15:32 Sean Neeley wrote: > > > I tried switching from Java 1.8 to Java 11 to see if that makes a > > difference. Now the VM Thread is using a lot less CPU: > > > > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ > COMMAND > > 2320 tomcat20 0 4659072 47872 19904 R 99.9 0.6 22:15.16 java > > 2326 tomcat20 0 4659072 47872 19904 R 4.6 0.6 0:56.43 VM > > Thread > > > > I tried running jstack on the processes, but I get this: > > > > 2320: Unable to open socket file: target process not responding or > HotSpot > > VM not loaded > > > > Did you attempt to run the command as the "Tomcat user"? > > BTW, Oracle recommends the use of "jcmd" over "jstack". Personally, I'd > give Mission Control/Flight Recorder a go. > I'm definitely running it as the tomcat user. I just tried jcmd with no arguments and the command completely hangs. The only way to terminate it is a kill -9. This seems almost like an OS level issue. We are opening a ticket with Red Hat support to see what they say.
Re: Having trouble with tomcat 7 installation on RHEL 7.8 power pc
On Wed, Jul 1, 2020, 15:32 Sean Neeley wrote: > I tried switching from Java 1.8 to Java 11 to see if that makes a > difference. Now the VM Thread is using a lot less CPU: > > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND > 2320 tomcat20 0 4659072 47872 19904 R 99.9 0.6 22:15.16 java > 2326 tomcat20 0 4659072 47872 19904 R 4.6 0.6 0:56.43 VM > Thread > > I tried running jstack on the processes, but I get this: > > 2320: Unable to open socket file: target process not responding or HotSpot > VM not loaded > Did you attempt to run the command as the "Tomcat user"? BTW, Oracle recommends the use of "jcmd" over "jstack". Personally, I'd give Mission Control/Flight Recorder a go.
Re: Problem with JarScanFilter, maybe a bug?
On 01/07/2020 20:28, Vitor Medina Cruz wrote: > On Wed, Jul 1, 2020 at 3:19 PM Mark Thomas wrote: > >> On 01/07/2020 18:09, Vitor Medina Cruz wrote: >>> On Wed, Jul 1, 2020 at 7:46 AM Mark Thomas wrote: >>> On 30/06/2020 14:19, Vitor Medina Cruz wrote: > Hello, > > I am trying to configure Tomcat in a way that it makes SCI scan only in > jars I explicitly specify to. I followed instructions from > https://tomcat.apache.org/tomcat-8.5-doc/config/jar-scan-filter.htm, >> in > both Tomcat 8 and 9, but with no success. I posted a question on > stackoverflow that explains more in detail what I did: > >> https://stackoverflow.com/questions/62602550/how-to-specify-which-classes-and-jars-gets-scanned-for-servlet-annotations-in-to > > And I also found other unanswered questions pointing to the same >> problem, > here is one example: > >> https://stackoverflow.com/questions/52876216/tomcat-too-slow-scanning-for-annotations > . > > The thing is that it is looking like an error to me because logs tells that > scanning is done as configured — if I add a jar for scanning in > JarScanFilter, the log show it is scanned, if I remove it, the log stop > reporting it's scanning — but after that, no matter what configuration >> I > made with JarScanFilter, the WebappServiceLoader loads servlet >> annotated > classes, such as @WebListener. The JarScanner machinery handles annotation and TLD scanning. WebappServiceLoader handles SCIs which are handled under the standard service loader mechanism. SCIs can load classes. > Any leads? Ideas? Anyone can confirm if that is an error or if I am >> using > the functionality wrongly or if I understand it wrongly. It looks like you aren't preventing the SCIs from being loaded. The specification isn't as clear as it could be here and there are still a few gaps. That is being worked on at Eclipse. A useful summary of the current position can be found at: >> https://github.com/apache/tomcat/blob/8.5.x/java/org/apache/catalina/startup/ContextConfig.java#L1092 The simplest way to block the Servlet 3 pluggability features is: 1. Add metadata-complete="true" to the web-app element in web.xml (disables annotation scanning for deploy time annotations - Servlet 3.1, 8.1) 2. Add to web.xml (disables any SCIs - Servlet 3.1, 8.2.2.d) Mark >>> Thanks. I, however, don't want to block all Servlet 3 pluggability as >> there >>> are frameworks already being made with no way of configuring it other >> than >>> that >> >> You can always explicitly define configuration in web.xml. >> >>> I would like to selectively choose which jars to be scanned in >>> order to avoid performance issues and rogue classes to be loaded. As is >>> seems, nor Servlet specification nor Tomcat in specific provides a way of >>> doing that, is that correct? >> >> No. >> >> Scanning != SCI loading. >> >> Scanning for deployment annotations can be controlled by the JarScanner. >> >> SCI loading can be controlled by an element that >> includes the JARs from which you do want to load SCIs. >> >> > But how can SCI loading takes place without scanning? That was what I > thought when I tried to control SCI loads, if I didn't scan any class at > all then no SCI should be loaded since no annotations will be found, but > that is not the case, so SCI loading must be doing an independent scanning > on it's own. No. SCIs are discovered via the service loader mechanism. Deployment annotations are discovered via JAR scanning. These are completely separate mechanisms. Note that an SCI may load and configure Servlets, Filters, Listeners etc. > In any way, leaving behind internal machinery, is it possible to define in > Tomcat which jars should be considered for annotation processing and SCI > loading, and which not? Yes. JarScanner for deployment annotations. for SCIs and @HandlesTypes matches. > I wanna tell Tomcat to only look and load for > @WebFiler, @WebListener, @WebServlet, @HandlesTypes and etc on specific > jars. Is that possible? @WebFiler, @WebListener and @WebServlet are deployment annotations so scanning for these is controlled by the JarScanner. If an SCI has an @HandlesTypes annotation then all JARs that are potential SCI sources will be scanned for matches. To put it another way, the JarScanner configuration does NOT control the search for @HandlesTypes matches. Any JAR eligible to provide an SCI will be scanned for @HandlesTypes. Those JARs are controlled by Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Having trouble with tomcat 7 installation on RHEL 7.8 power pc
I tried switching from Java 1.8 to Java 11 to see if that makes a difference. Now the VM Thread is using a lot less CPU: PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 2320 tomcat20 0 4659072 47872 19904 R 99.9 0.6 22:15.16 java 2326 tomcat20 0 4659072 47872 19904 R 4.6 0.6 0:56.43 VM Thread I tried running jstack on the processes, but I get this: 2320: Unable to open socket file: target process not responding or HotSpot VM not loaded I tried running strace on the process and see this over and over in the output: [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, [pid 2334] <... futex resumed>)= -1 ETIMEDOUT (Connection timed out) [pid 2334] futex(0x3fff7c1fd228, FUTEX_WAKE_PRIVATE, 1) = 0 [pid 2334] futex(0x3fff7c1fd254, FUTEX_WAIT_BITSET_PRIVATE, 17307, {tv_sec=2074, tv_nsec=141673818}, 0x [pid 2326] <... nanosleep resumed>NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 [pid 2326] nanosleep({tv_sec=0, tv_nsec=100}, NULL) = 0 /var/log/tomcat logs are still empty. I'm running out of ideas. Sean On Wed, Jul 1, 2020 at 12:55 PM wrote: > Sean, > > > > -Original Message- > > From: Sean Neeley > > Sent: Wednesday, July 01, 2020 12:26 PM > > To: Tomcat Users List > > Subject: Re: Having trouble with tomcat 7 installation on RHEL 7.8 power > pc > > > > John, The top two processes are: > > > > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ > COMMAND > > 19360 tomcat20 0 3302144 42560 18944 R 99.9 0.5 46:02.37 java > > 19363 tomcat20 0 3302144 42560 18944 R 98.4 0.5 45:48.50 VM > Thread > > > > I tried kill -3 on both of them, plus the java process ID I see from a > "ps" > > command, several seconds apart. The only logs created are these, and > they > > are still all empty after the kill -3: > > -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 catalina.2020-07-01.log > > -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 > host-manager.2020-07-01.log > > -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 localhost.2020-07-01.log > > -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 manager.2020-07-01.log > > > > Any other ideas? > > > > On Wed, Jul 1, 2020 at 12:09 PM > > wrote: > > > > > Sean, > > > > > > > -Original Message- > > > > From: Sean Neeley > > > > Sent: Wednesday, July 01, 2020 11:15 AM > > > > To: users@tomcat.apache.org > > > > Subject: Having trouble with tomcat 7 installation on RHEL 7.8 power > > > > pc > > > > > > > > I just installed tomcat 7 on a Red Hat Enterprise Linux Server 7.8, > > > power pc > > > > system. As soon as the service starts, the java process uses 100% > cpu. > > > > Logs get created in /var/log/tomcat, but they all have size 0 bytes. > > > > I > > > have not > > > > modified the standard configuration (tomcat.conf, server.xml, etc). > > > > The tomcat packages that are installed are: > > > > > > > > tomcat-7.0.76-12.el7_8.noarch > > > > tomcat-jsp-2.2-api-7.0.76-12.el7_8.noarch > > > > tomcat-el-2.2-api-7.0.76-12.el7_8.noarch > > > > tomcat-lib-7.0.76-12.el7_8.noarch > > > > tomcat-servlet-3.0-api-7.0.76-12.el7_8.noarch > > > > > > > > Are there any tricks I can use to troubleshoot what is going wrong? > > > > Without any logs being created, it seems impossible to determine the > > > > problem. Thanks for helping. > > > > > > > > Sean > > > > > > Use kill -3 to take several thread dumps about 5-10 seconds apart. > > > They will probably end up in catalina.out. > > > > > > You can also use top -H to see what individual threads are using CPU. > > > > > > > > Sorry, I don't know much more. > > It's interesting that the CPU is being consumed by "VM Thread." It > doesn't look like an ordinary Tomcat or application problem. When I > googled "java vm thread," I got some interesting results, such as this one: > > > https://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/hangloop.html > > That also describes the scenario where a thread dump doesn't seem to do > anything. > > >
Re: Problem with JarScanFilter, maybe a bug?
On Wed, Jul 1, 2020 at 3:19 PM Mark Thomas wrote: > On 01/07/2020 18:09, Vitor Medina Cruz wrote: > > On Wed, Jul 1, 2020 at 7:46 AM Mark Thomas wrote: > > > >> On 30/06/2020 14:19, Vitor Medina Cruz wrote: > >>> Hello, > >>> > >>> I am trying to configure Tomcat in a way that it makes SCI scan only in > >>> jars I explicitly specify to. I followed instructions from > >>> https://tomcat.apache.org/tomcat-8.5-doc/config/jar-scan-filter.htm, > in > >>> both Tomcat 8 and 9, but with no success. I posted a question on > >>> stackoverflow that explains more in detail what I did: > >>> > >> > https://stackoverflow.com/questions/62602550/how-to-specify-which-classes-and-jars-gets-scanned-for-servlet-annotations-in-to > >>> > >>> And I also found other unanswered questions pointing to the same > problem, > >>> here is one example: > >>> > >> > https://stackoverflow.com/questions/52876216/tomcat-too-slow-scanning-for-annotations > >>> . > >>> > >>> The thing is that it is looking like an error to me because logs tells > >> that > >>> scanning is done as configured — if I add a jar for scanning in > >>> JarScanFilter, the log show it is scanned, if I remove it, the log stop > >>> reporting it's scanning — but after that, no matter what configuration > I > >>> made with JarScanFilter, the WebappServiceLoader loads servlet > annotated > >>> classes, such as @WebListener. > >> > >> The JarScanner machinery handles annotation and TLD scanning. > >> > >> WebappServiceLoader handles SCIs which are handled under the standard > >> service loader mechanism. SCIs can load classes. > >> > >>> Any leads? Ideas? Anyone can confirm if that is an error or if I am > using > >>> the functionality wrongly or if I understand it wrongly. > >> > >> It looks like you aren't preventing the SCIs from being loaded. > >> > >> The specification isn't as clear as it could be here and there are still > >> a few gaps. That is being worked on at Eclipse. A useful summary of the > >> current position can be found at: > >> > >> > >> > https://github.com/apache/tomcat/blob/8.5.x/java/org/apache/catalina/startup/ContextConfig.java#L1092 > >> > >> The simplest way to block the Servlet 3 pluggability features is: > >> > >> 1. Add metadata-complete="true" to the web-app element in web.xml > >>(disables annotation scanning for deploy time annotations - > >> Servlet 3.1, 8.1) > >> > >> 2. Add to web.xml > >>(disables any SCIs - Servlet 3.1, 8.2.2.d) > >> > >> Mark > >> > >> > > Thanks. I, however, don't want to block all Servlet 3 pluggability as > there > > are frameworks already being made with no way of configuring it other > than > > that > > You can always explicitly define configuration in web.xml. > > > I would like to selectively choose which jars to be scanned in > > order to avoid performance issues and rogue classes to be loaded. As is > > seems, nor Servlet specification nor Tomcat in specific provides a way of > > doing that, is that correct? > > No. > > Scanning != SCI loading. > > Scanning for deployment annotations can be controlled by the JarScanner. > > SCI loading can be controlled by an element that > includes the JARs from which you do want to load SCIs. > > But how can SCI loading takes place without scanning? That was what I thought when I tried to control SCI loads, if I didn't scan any class at all then no SCI should be loaded since no annotations will be found, but that is not the case, so SCI loading must be doing an independent scanning on it's own. In any way, leaving behind internal machinery, is it possible to define in Tomcat which jars should be considered for annotation processing and SCI loading, and which not? I wanna tell Tomcat to only look and load for @WebFiler, @WebListener, @WebServlet, @HandlesTypes and etc on specific jars. Is that possible? Regards, Vitor > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Problem with JarScanFilter, maybe a bug?
On 01/07/2020 18:09, Vitor Medina Cruz wrote: > On Wed, Jul 1, 2020 at 7:46 AM Mark Thomas wrote: > >> On 30/06/2020 14:19, Vitor Medina Cruz wrote: >>> Hello, >>> >>> I am trying to configure Tomcat in a way that it makes SCI scan only in >>> jars I explicitly specify to. I followed instructions from >>> https://tomcat.apache.org/tomcat-8.5-doc/config/jar-scan-filter.htm, in >>> both Tomcat 8 and 9, but with no success. I posted a question on >>> stackoverflow that explains more in detail what I did: >>> >> https://stackoverflow.com/questions/62602550/how-to-specify-which-classes-and-jars-gets-scanned-for-servlet-annotations-in-to >>> >>> And I also found other unanswered questions pointing to the same problem, >>> here is one example: >>> >> https://stackoverflow.com/questions/52876216/tomcat-too-slow-scanning-for-annotations >>> . >>> >>> The thing is that it is looking like an error to me because logs tells >> that >>> scanning is done as configured — if I add a jar for scanning in >>> JarScanFilter, the log show it is scanned, if I remove it, the log stop >>> reporting it's scanning — but after that, no matter what configuration I >>> made with JarScanFilter, the WebappServiceLoader loads servlet annotated >>> classes, such as @WebListener. >> >> The JarScanner machinery handles annotation and TLD scanning. >> >> WebappServiceLoader handles SCIs which are handled under the standard >> service loader mechanism. SCIs can load classes. >> >>> Any leads? Ideas? Anyone can confirm if that is an error or if I am using >>> the functionality wrongly or if I understand it wrongly. >> >> It looks like you aren't preventing the SCIs from being loaded. >> >> The specification isn't as clear as it could be here and there are still >> a few gaps. That is being worked on at Eclipse. A useful summary of the >> current position can be found at: >> >> >> https://github.com/apache/tomcat/blob/8.5.x/java/org/apache/catalina/startup/ContextConfig.java#L1092 >> >> The simplest way to block the Servlet 3 pluggability features is: >> >> 1. Add metadata-complete="true" to the web-app element in web.xml >>(disables annotation scanning for deploy time annotations - >> Servlet 3.1, 8.1) >> >> 2. Add to web.xml >>(disables any SCIs - Servlet 3.1, 8.2.2.d) >> >> Mark >> >> > Thanks. I, however, don't want to block all Servlet 3 pluggability as there > are frameworks already being made with no way of configuring it other than > that You can always explicitly define configuration in web.xml. > I would like to selectively choose which jars to be scanned in > order to avoid performance issues and rogue classes to be loaded. As is > seems, nor Servlet specification nor Tomcat in specific provides a way of > doing that, is that correct? No. Scanning != SCI loading. Scanning for deployment annotations can be controlled by the JarScanner. SCI loading can be controlled by an element that includes the JARs from which you do want to load SCIs. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Having trouble with tomcat 7 installation on RHEL 7.8 power pc
Sean, > -Original Message- > From: Sean Neeley > Sent: Wednesday, July 01, 2020 12:26 PM > To: Tomcat Users List > Subject: Re: Having trouble with tomcat 7 installation on RHEL 7.8 power pc > > John, The top two processes are: > > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND > 19360 tomcat20 0 3302144 42560 18944 R 99.9 0.5 46:02.37 java > 19363 tomcat20 0 3302144 42560 18944 R 98.4 0.5 45:48.50 VM Thread > > I tried kill -3 on both of them, plus the java process ID I see from a "ps" > command, several seconds apart. The only logs created are these, and they > are still all empty after the kill -3: > -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 catalina.2020-07-01.log > -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 host-manager.2020-07-01.log > -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 localhost.2020-07-01.log > -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 manager.2020-07-01.log > > Any other ideas? > > On Wed, Jul 1, 2020 at 12:09 PM > wrote: > > > Sean, > > > > > -Original Message- > > > From: Sean Neeley > > > Sent: Wednesday, July 01, 2020 11:15 AM > > > To: users@tomcat.apache.org > > > Subject: Having trouble with tomcat 7 installation on RHEL 7.8 power > > > pc > > > > > > I just installed tomcat 7 on a Red Hat Enterprise Linux Server 7.8, > > power pc > > > system. As soon as the service starts, the java process uses 100% cpu. > > > Logs get created in /var/log/tomcat, but they all have size 0 bytes. > > > I > > have not > > > modified the standard configuration (tomcat.conf, server.xml, etc). > > > The tomcat packages that are installed are: > > > > > > tomcat-7.0.76-12.el7_8.noarch > > > tomcat-jsp-2.2-api-7.0.76-12.el7_8.noarch > > > tomcat-el-2.2-api-7.0.76-12.el7_8.noarch > > > tomcat-lib-7.0.76-12.el7_8.noarch > > > tomcat-servlet-3.0-api-7.0.76-12.el7_8.noarch > > > > > > Are there any tricks I can use to troubleshoot what is going wrong? > > > Without any logs being created, it seems impossible to determine the > > > problem. Thanks for helping. > > > > > > Sean > > > > Use kill -3 to take several thread dumps about 5-10 seconds apart. > > They will probably end up in catalina.out. > > > > You can also use top -H to see what individual threads are using CPU. > > > > Sorry, I don't know much more. It's interesting that the CPU is being consumed by "VM Thread." It doesn't look like an ordinary Tomcat or application problem. When I googled "java vm thread," I got some interesting results, such as this one: https://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/hangloop.html That also describes the scenario where a thread dump doesn't seem to do anything.
Re: Having trouble with tomcat 7 installation on RHEL 7.8 power pc
John, The top two processes are: PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 19360 tomcat20 0 3302144 42560 18944 R 99.9 0.5 46:02.37 java 19363 tomcat20 0 3302144 42560 18944 R 98.4 0.5 45:48.50 VM Thread I tried kill -3 on both of them, plus the java process ID I see from a "ps" command, several seconds apart. The only logs created are these, and they are still all empty after the kill -3: -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 catalina.2020-07-01.log -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 host-manager.2020-07-01.log -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 localhost.2020-07-01.log -rw-r--r-- 1 tomcat tomcat0 Jul 1 10:22 manager.2020-07-01.log Any other ideas? On Wed, Jul 1, 2020 at 12:09 PM wrote: > Sean, > > > -Original Message- > > From: Sean Neeley > > Sent: Wednesday, July 01, 2020 11:15 AM > > To: users@tomcat.apache.org > > Subject: Having trouble with tomcat 7 installation on RHEL 7.8 power pc > > > > I just installed tomcat 7 on a Red Hat Enterprise Linux Server 7.8, > power pc > > system. As soon as the service starts, the java process uses 100% cpu. > > Logs get created in /var/log/tomcat, but they all have size 0 bytes. I > have not > > modified the standard configuration (tomcat.conf, server.xml, etc). The > > tomcat packages that are installed are: > > > > tomcat-7.0.76-12.el7_8.noarch > > tomcat-jsp-2.2-api-7.0.76-12.el7_8.noarch > > tomcat-el-2.2-api-7.0.76-12.el7_8.noarch > > tomcat-lib-7.0.76-12.el7_8.noarch > > tomcat-servlet-3.0-api-7.0.76-12.el7_8.noarch > > > > Are there any tricks I can use to troubleshoot what is going wrong? > > Without any logs being created, it seems impossible to determine the > > problem. Thanks for helping. > > > > Sean > > Use kill -3 to take several thread dumps about 5-10 seconds apart. They > will probably end up in catalina.out. > > You can also use top -H to see what individual threads are using CPU. > >
Re: Having trouble with tomcat 7 installation on RHEL 7.8 power pc
Permissions of /var/log/tomcat look fine: drwxrwx---. 2 tomcat root 4096 Jul 1 08:34 tomcat And it is creating the logs, they are just empty: -rw-r--r-- 1 tomcat tomcat0 Jul 1 08:34 catalina.2020-07-01.log -rw-r--r-- 1 tomcat tomcat0 Jul 1 08:34 host-manager.2020-07-01.log -rw-r--r-- 1 tomcat tomcat0 Jul 1 08:34 localhost.2020-07-01.log -rw-r--r-- 1 tomcat tomcat0 Jul 1 08:34 manager.2020-07-01.log /var/log/messages shows: Jul 1 09:31:03 ecom-main systemd: Started Apache Tomcat Web Application Container. Jul 1 09:31:03 ecom-main server: Java virtual machine used: /usr/lib/jvm/jre/bin/java Jul 1 09:31:03 ecom-main server: classpath used: /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar Jul 1 09:31:03 ecom-main server: main class used: org.apache.catalina.startup.Bootstrap Jul 1 09:31:03 ecom-main server: flags used: Jul 1 09:31:03 ecom-main server: options used: -Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager Jul 1 09:31:03 ecom-main server: arguments used: start On Wed, Jul 1, 2020 at 12:00 PM calder wrote: > On Wed, Jul 1, 2020, 11:15 Sean Neeley wrote: > > > I just installed tomcat 7 on a Red Hat Enterprise Linux Server 7.8, power > > pc system. As soon as the service starts, the java process uses 100% > cpu. > > Logs get created in /var/log/tomcat, but they all have size 0 bytes. I > > have not modified the standard configuration (tomcat.conf, server.xml, > > etc). The tomcat packages that are installed are: > > > > tomcat-7.0.76-12.el7_8.noarch > > tomcat-jsp-2.2-api-7.0.76-12.el7_8.noarch > > tomcat-el-2.2-api-7.0.76-12.el7_8.noarch > > tomcat-lib-7.0.76-12.el7_8.noarch > > tomcat-servlet-3.0-api-7.0.76-12.el7_8.noarch > > > > Are there any tricks I can use to troubleshoot what is going wrong? > > > Check "syslog" for clues. > > Also, double-check the permissions for /var/log/tomcat. >
RE: Having trouble with tomcat 7 installation on RHEL 7.8 power pc
Sean, > -Original Message- > From: Sean Neeley > Sent: Wednesday, July 01, 2020 11:15 AM > To: users@tomcat.apache.org > Subject: Having trouble with tomcat 7 installation on RHEL 7.8 power pc > > I just installed tomcat 7 on a Red Hat Enterprise Linux Server 7.8, power pc > system. As soon as the service starts, the java process uses 100% cpu. > Logs get created in /var/log/tomcat, but they all have size 0 bytes. I have > not > modified the standard configuration (tomcat.conf, server.xml, etc). The > tomcat packages that are installed are: > > tomcat-7.0.76-12.el7_8.noarch > tomcat-jsp-2.2-api-7.0.76-12.el7_8.noarch > tomcat-el-2.2-api-7.0.76-12.el7_8.noarch > tomcat-lib-7.0.76-12.el7_8.noarch > tomcat-servlet-3.0-api-7.0.76-12.el7_8.noarch > > Are there any tricks I can use to troubleshoot what is going wrong? > Without any logs being created, it seems impossible to determine the > problem. Thanks for helping. > > Sean Use kill -3 to take several thread dumps about 5-10 seconds apart. They will probably end up in catalina.out. You can also use top -H to see what individual threads are using CPU.
Re: Problem with JarScanFilter, maybe a bug?
On Wed, Jul 1, 2020 at 7:46 AM Mark Thomas wrote: > On 30/06/2020 14:19, Vitor Medina Cruz wrote: > > Hello, > > > > I am trying to configure Tomcat in a way that it makes SCI scan only in > > jars I explicitly specify to. I followed instructions from > > https://tomcat.apache.org/tomcat-8.5-doc/config/jar-scan-filter.htm, in > > both Tomcat 8 and 9, but with no success. I posted a question on > > stackoverflow that explains more in detail what I did: > > > https://stackoverflow.com/questions/62602550/how-to-specify-which-classes-and-jars-gets-scanned-for-servlet-annotations-in-to > > > > And I also found other unanswered questions pointing to the same problem, > > here is one example: > > > https://stackoverflow.com/questions/52876216/tomcat-too-slow-scanning-for-annotations > > . > > > > The thing is that it is looking like an error to me because logs tells > that > > scanning is done as configured — if I add a jar for scanning in > > JarScanFilter, the log show it is scanned, if I remove it, the log stop > > reporting it's scanning — but after that, no matter what configuration I > > made with JarScanFilter, the WebappServiceLoader loads servlet annotated > > classes, such as @WebListener. > > The JarScanner machinery handles annotation and TLD scanning. > > WebappServiceLoader handles SCIs which are handled under the standard > service loader mechanism. SCIs can load classes. > > > Any leads? Ideas? Anyone can confirm if that is an error or if I am using > > the functionality wrongly or if I understand it wrongly. > > It looks like you aren't preventing the SCIs from being loaded. > > The specification isn't as clear as it could be here and there are still > a few gaps. That is being worked on at Eclipse. A useful summary of the > current position can be found at: > > > https://github.com/apache/tomcat/blob/8.5.x/java/org/apache/catalina/startup/ContextConfig.java#L1092 > > The simplest way to block the Servlet 3 pluggability features is: > > 1. Add metadata-complete="true" to the web-app element in web.xml >(disables annotation scanning for deploy time annotations - > Servlet 3.1, 8.1) > > 2. Add to web.xml >(disables any SCIs - Servlet 3.1, 8.2.2.d) > > Mark > > Thanks. I, however, don't want to block all Servlet 3 pluggability as there are frameworks already being made with no way of configuring it other than that I would like to selectively choose which jars to be scanned in order to avoid performance issues and rogue classes to be loaded. As is seems, nor Servlet specification nor Tomcat in specific provides a way of doing that, is that correct? > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Having trouble with tomcat 7 installation on RHEL 7.8 power pc
On Wed, Jul 1, 2020, 11:15 Sean Neeley wrote: > I just installed tomcat 7 on a Red Hat Enterprise Linux Server 7.8, power > pc system. As soon as the service starts, the java process uses 100% cpu. > Logs get created in /var/log/tomcat, but they all have size 0 bytes. I > have not modified the standard configuration (tomcat.conf, server.xml, > etc). The tomcat packages that are installed are: > > tomcat-7.0.76-12.el7_8.noarch > tomcat-jsp-2.2-api-7.0.76-12.el7_8.noarch > tomcat-el-2.2-api-7.0.76-12.el7_8.noarch > tomcat-lib-7.0.76-12.el7_8.noarch > tomcat-servlet-3.0-api-7.0.76-12.el7_8.noarch > > Are there any tricks I can use to troubleshoot what is going wrong? Check "syslog" for clues. Also, double-check the permissions for /var/log/tomcat.
Having trouble with tomcat 7 installation on RHEL 7.8 power pc
I just installed tomcat 7 on a Red Hat Enterprise Linux Server 7.8, power pc system. As soon as the service starts, the java process uses 100% cpu. Logs get created in /var/log/tomcat, but they all have size 0 bytes. I have not modified the standard configuration (tomcat.conf, server.xml, etc). The tomcat packages that are installed are: tomcat-7.0.76-12.el7_8.noarch tomcat-jsp-2.2-api-7.0.76-12.el7_8.noarch tomcat-el-2.2-api-7.0.76-12.el7_8.noarch tomcat-lib-7.0.76-12.el7_8.noarch tomcat-servlet-3.0-api-7.0.76-12.el7_8.noarch Are there any tricks I can use to troubleshoot what is going wrong? Without any logs being created, it seems impossible to determine the problem. Thanks for helping. Sean
Re: Problem with JarScanFilter, maybe a bug?
On 30/06/2020 14:19, Vitor Medina Cruz wrote: > Hello, > > I am trying to configure Tomcat in a way that it makes SCI scan only in > jars I explicitly specify to. I followed instructions from > https://tomcat.apache.org/tomcat-8.5-doc/config/jar-scan-filter.htm, in > both Tomcat 8 and 9, but with no success. I posted a question on > stackoverflow that explains more in detail what I did: > https://stackoverflow.com/questions/62602550/how-to-specify-which-classes-and-jars-gets-scanned-for-servlet-annotations-in-to > > And I also found other unanswered questions pointing to the same problem, > here is one example: > https://stackoverflow.com/questions/52876216/tomcat-too-slow-scanning-for-annotations > . > > The thing is that it is looking like an error to me because logs tells that > scanning is done as configured — if I add a jar for scanning in > JarScanFilter, the log show it is scanned, if I remove it, the log stop > reporting it's scanning — but after that, no matter what configuration I > made with JarScanFilter, the WebappServiceLoader loads servlet annotated > classes, such as @WebListener. The JarScanner machinery handles annotation and TLD scanning. WebappServiceLoader handles SCIs which are handled under the standard service loader mechanism. SCIs can load classes. > Any leads? Ideas? Anyone can confirm if that is an error or if I am using > the functionality wrongly or if I understand it wrongly. It looks like you aren't preventing the SCIs from being loaded. The specification isn't as clear as it could be here and there are still a few gaps. That is being worked on at Eclipse. A useful summary of the current position can be found at: https://github.com/apache/tomcat/blob/8.5.x/java/org/apache/catalina/startup/ContextConfig.java#L1092 The simplest way to block the Servlet 3 pluggability features is: 1. Add metadata-complete="true" to the web-app element in web.xml (disables annotation scanning for deploy time annotations - Servlet 3.1, 8.1) 2. Add to web.xml (disables any SCIs - Servlet 3.1, 8.2.2.d) Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat session replication
Am 1. Juli 2020 12:21:46 MESZ schrieb Mark Thomas : >On 01/07/2020 11:19, Thomas Meyer wrote: >> Am 30. Juni 2020 11:07:36 MESZ schrieb Mark Thomas >: >>> On 29/06/2020 21:41, Christopher Schultz wrote: Mark, On 6/27/20 05:29, Mark Thomas wrote: > On 27/06/2020 10:19, Thomas Meyer wrote: >> Hi, >> >> A few questions regarding tomcat session replication: > load-balancing and session replication are two separate parts of > an overall clustering solution. >> 1) is the jvmRoute attribute on Engine object necessary for >> session replication to work correctly? > No, but if you don't use it it places a number of restrictions on > the web application behaviour and on the configuration of session > replication. > The limitations are: - you need to use the DeltaManager (which > doesn't scale as well as the BackupManager); - any requests made >by > the client that depend on the session MUST be issued in series, >not > in parallel; and This is only true of requests that would modify the session-state >in >>> a way that needed to be deterministic, right? A bunch of GET requests that don't change the session ought to be okay in parallel (as long >>> as any prior state-changing requests have completed _ those changes replicated). >>> >>> Yes. >>> You don't want state changes in parallel on different nodes. >>> Any request that depends on a previous change in state can't be >issued >>> until the state changing request has completed and the changes >>> replicated. >>> > - the session Manager must be configured to update all the other > nodes in the cluster BEFORE the current request returns to the > client. Same (negative) caveat here, right? >>> >>> Yes. >>> >>> Essentially you want channelSendOptions="6". >> >> Hi, >> >> Yes I'm using that option. But it still gives an error, but I may now >found some hints what's going wrong: >> >> When using Spring's ChangeSessionIdAuthStrategy it fails with unknown >CSRF token. >> >> It looks like the node fails to replicate, i.e. doesn't export, the >session data after a changeSessionId call. >> >> When using Spring's SessionFixationProtectionStrategy (which >basically creates a new session and copy all attributes to the new >session) it works correctly with tomcats session replication. >> >> So it looks like calling changeSessionId fails to somehow replication >the new session state to the remote nodes. >> >> Looking at ManagerBase "session" attribute it's unclear if it >contains only "internal session IDs" or external session IDs which do >change. >> >> The ReplicationValve seems to call manager.findSession with the >internal ID. >> >> Maybe somewhere something mixes up internal and external session IDs >or forgets to update ManagerBase.session map. >> >> Opinions? > >Maybe this: >https://bz.apache.org/bugzilla/show_bug.cgi?id=64560 Yes, that's seems to be exactly the same problem! And it's already fixed! Thank you very much! I'll update our tomcat version from 9.0.34 to the fixed version. Regards Thomas - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Fwd: Reverse proxy and SSL redirect
On 01/07/2020 00:41, rugman66 . wrote: > On Wed, Apr 22, 2020 at 9:21 AM Mark Thomas wrote: >> >> On 22/04/2020 00:11, rugman66 . wrote: >> >> >> >>>Tomcat log (I'm trying to get more debug level logging) >>> 2020-04-21 13:39:33 INFO app.CompletionRestController >>> Unsupported Media Type in Header >>> >>> Postman >>>415 Unsupported Media Type >>> >>> GET URL >>> http://server.com/app/api/completions.json?username=foo >>> >>> Both Tomcat and Apache are running SSL because all internal endpoints >>> are required to be secure. >> >> Looks like the app is generating the error. That moves us forwards. >> >> Try enabling the RequestDumperFilter. That should dump the full set of >> request headers received which will hopefully help explain what is going on. >> >> Mark > > Hi Mark, > > Was on unplanned leave for the past few months, but back. > > I did try to enable RequestDumperFilter, however the file was created > but no log entries created. I did find something interesting. When I > test in Postman with > HTTP it does redirect to HTTPD but throws the error. However when I > change the URL in Postman using HTTPD I get the expected reply and see > the > proxy is indeed working. It's only throwing the error when the > redirect occurs. Seems to me the issue lies there, but I still can't > find a resolution. Any > suggestions would be appreciated. You need to find a way to see the full traffic for both client<->httpd and httpd<->Tomcat. Wireshark is one option. You'll need to configure it to decrypt the TLS. The access logs will also confirm whether requests are passed to Tomcat or handled by httpd. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat session replication
On 01/07/2020 11:19, Thomas Meyer wrote: > Am 30. Juni 2020 11:07:36 MESZ schrieb Mark Thomas : >> On 29/06/2020 21:41, Christopher Schultz wrote: >>> Mark, >>> >>> On 6/27/20 05:29, Mark Thomas wrote: On 27/06/2020 10:19, Thomas Meyer wrote: > Hi, > > A few questions regarding tomcat session replication: >>> load-balancing and session replication are two separate parts of an overall clustering solution. >>> > 1) is the jvmRoute attribute on Engine object necessary for > session replication to work correctly? >>> No, but if you don't use it it places a number of restrictions on the web application behaviour and on the configuration of session replication. >>> The limitations are: - you need to use the DeltaManager (which doesn't scale as well as the BackupManager); - any requests made by the client that depend on the session MUST be issued in series, not in parallel; and >>> >>> This is only true of requests that would modify the session-state in >> a >>> way that needed to be deterministic, right? A bunch of GET requests >>> that don't change the session ought to be okay in parallel (as long >> as >>> any prior state-changing requests have completed _ those changes >>> replicated). >> >> Yes. >> You don't want state changes in parallel on different nodes. >> Any request that depends on a previous change in state can't be issued >> until the state changing request has completed and the changes >> replicated. >> - the session Manager must be configured to update all the other nodes in the cluster BEFORE the current request returns to the client. >>> >>> Same (negative) caveat here, right? >> >> Yes. >> >> Essentially you want channelSendOptions="6". > > Hi, > > Yes I'm using that option. But it still gives an error, but I may now found > some hints what's going wrong: > > When using Spring's ChangeSessionIdAuthStrategy it fails with unknown CSRF > token. > > It looks like the node fails to replicate, i.e. doesn't export, the session > data after a changeSessionId call. > > When using Spring's SessionFixationProtectionStrategy (which basically > creates a new session and copy all attributes to the new session) it works > correctly with tomcats session replication. > > So it looks like calling changeSessionId fails to somehow replication the new > session state to the remote nodes. > > Looking at ManagerBase "session" attribute it's unclear if it contains only > "internal session IDs" or external session IDs which do change. > > The ReplicationValve seems to call manager.findSession with the internal ID. > > Maybe somewhere something mixes up internal and external session IDs or > forgets to update ManagerBase.session map. > > Opinions? Maybe this: https://bz.apache.org/bugzilla/show_bug.cgi?id=64560 Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat session replication
Am 30. Juni 2020 11:07:36 MESZ schrieb Mark Thomas : >On 29/06/2020 21:41, Christopher Schultz wrote: >> Mark, >> >> On 6/27/20 05:29, Mark Thomas wrote: >>> On 27/06/2020 10:19, Thomas Meyer wrote: Hi, A few questions regarding tomcat session replication: >> >>> load-balancing and session replication are two separate parts of >>> an overall clustering solution. >> 1) is the jvmRoute attribute on Engine object necessary for session replication to work correctly? >> >>> No, but if you don't use it it places a number of restrictions on >>> the web application behaviour and on the configuration of session >>> replication. >> >>> The limitations are: - you need to use the DeltaManager (which >>> doesn't scale as well as the BackupManager); - any requests made by >>> the client that depend on the session MUST be issued in series, not >>> in parallel; and >> >> This is only true of requests that would modify the session-state in >a >> way that needed to be deterministic, right? A bunch of GET requests >> that don't change the session ought to be okay in parallel (as long >as >> any prior state-changing requests have completed _ those changes >> replicated). > >Yes. >You don't want state changes in parallel on different nodes. >Any request that depends on a previous change in state can't be issued >until the state changing request has completed and the changes >replicated. > >>> - the session Manager must be configured to update all the other >>> nodes in the cluster BEFORE the current request returns to the >>> client. >> >> Same (negative) caveat here, right? > >Yes. > >Essentially you want channelSendOptions="6". Hi, Yes I'm using that option. But it still gives an error, but I may now found some hints what's going wrong: When using Spring's ChangeSessionIdAuthStrategy it fails with unknown CSRF token. It looks like the node fails to replicate, i.e. doesn't export, the session data after a changeSessionId call. When using Spring's SessionFixationProtectionStrategy (which basically creates a new session and copy all attributes to the new session) it works correctly with tomcats session replication. So it looks like calling changeSessionId fails to somehow replication the new session state to the remote nodes. Looking at ManagerBase "session" attribute it's unclear if it contains only "internal session IDs" or external session IDs which do change. The ReplicationValve seems to call manager.findSession with the internal ID. Maybe somewhere something mixes up internal and external session IDs or forgets to update ManagerBase.session map. Opinions? - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
SameSite attribute handling
Hi All, We can add the samesite attribute in set-cookie header through context.xml entry in tomcat. Is there any other way, can we add samesite attribute in response of set-cookie header. Context changes reflecting issue in tenable vulnerable. Hence looking for any other way. I tried with filter by using setHeader but it is sending two set-Cookie header. Regards, Abirami.S