Re: client write waits on postgresql RDS

2021-06-15 Thread Ayub Khan
quot;,defaultValue ="0") String d) { Object menuList = new Object(); try { menuList = menuManagmentService.listAll(a, b, c, d); //this service calls the dao in the service class } catch (Exception e) { LOG.error(e.getLocalizedMessage(), e); } return me

Re: client write waits on postgresql RDS

2021-06-15 Thread Ayub Khan
Christopher Schultz < ch...@christopherschultz.net> wrote: > Ayub, > > On 6/15/21 11:33, Ayub Khan wrote: > > Would it be a cursor issue, as there seems to be how cursors are handled > in > > postgres and Oracle database. Below are the steps we take from jdbc side >

Re: client write waits on postgresql RDS

2021-06-15 Thread Ayub Khan
. set autocommit to true 7. close the resultset,callable statement and connection On Thu, Jun 10, 2021 at 8:10 AM Ayub Khan wrote: > Seeing client write waits on postgresql as attached in the image. Is there > any bottle neck which is causing the client write waits on postgresql ? >

Re: client write waits on postgresql RDS

2021-06-14 Thread Ayub Khan
Chris, In jprofiler I see that the threads are waiting on reading the response from database side On Thu, 10 Jun 2021, 08:10 Ayub Khan, wrote: > Seeing client write waits on postgresql as attached in the image. Is there > any bottle neck which is causing the client write waits on post

Re: client write waits on postgresql RDS

2021-06-11 Thread Ayub Khan
lableStatement.close(); if (con != null) con.close(); } return menuMobileListCombo; } On Thu, Jun 10, 2021 at 8:10 AM Ayub Khan wrote: > Seeing client write waits on postgresql as attached in the image. Is there > any bottle neck which is causing the client write waits on postgresql ? &

client write waits on postgresql RDS

2021-06-09 Thread Ayub Khan
Seeing client write waits on postgresql as attached in the image. Is there any bottle neck which is causing the client write waits on postgresql ? Below is the test setup Jmeter-->(load balanced tomcat on ec2 instances)>rds read replicas All these are running on different ec2 instances in

Re: NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-11-13 Thread Ayub Khan
11/12/20 11:20, Ayub Khan wrote: > > Chris, > > > > That's correct, it's just a plain static hello world page I created to > > verify tomcat. It is served by tomcat. I have bundled this page in the > same > > context where the service is running. When I create

Re: NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-11-12 Thread Ayub Khan
not even serve the html page On Thu, Nov 12, 2020 at 7:01 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > Ayub, > > On 11/12/20 10:47, Ayub Khan wrote: > > Chris, > > > > I am using hikaricp connection pooling and the maximum pool size is set > to

Re: NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-11-12 Thread Ayub Khan
--> tomcat on the same VM and tomcat consumed total memory of 30Gig and CPU was at 28% t On Thu, Nov 12, 2020 at 6:47 PM Ayub Khan wrote: > Chris, > > I am using hikaricp connection pooling and the maximum pool size is set to > 100, without specifying minimum idle connections. Even dur

Re: NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-11-12 Thread Ayub Khan
> Ayub, > > On 11/11/20 16:16, Ayub Khan wrote: > > I was load testing using the ec2 load balancer dns. I have increased the > > connector timeout to 6000 and also gave 32gig to the JVM of tomcat. I am > > not seeing connection timeout in nginx logs now. No errors in kernel.log

Re: NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-11-12 Thread Ayub Khan
4:05 PM Martin Grigorov wrote: > On Thu, Nov 12, 2020 at 2:40 PM Ayub Khan wrote: > > > Martin, > > > > Could you provide me a command which you want me to run and provide you > the > > results which might help you to debug this issue ? > > > > 1) star

Re: NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-11-12 Thread Ayub Khan
Martin, Could you provide me a command which you want me to run and provide you the results which might help you to debug this issue ? On Thu, Nov 12, 2020 at 1:36 PM Martin Grigorov wrote: > On Thu, Nov 12, 2020 at 10:37 AM Ayub Khan wrote: > > > Martin, > > > > Th

Re: NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-11-12 Thread Ayub Khan
/proc/`cat /var/run/tomcat8.pid`/fd/ | wc -l" On Thu, Nov 12, 2020 at 10:56 AM Martin Grigorov wrote: > On Wed, Nov 11, 2020 at 11:17 PM Ayub Khan wrote: > > > Chris, > > > > I was load testing using the ec2 load balancer dns. I have increased the > > conne

Re: NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-11-11 Thread Ayub Khan
n 11/3/20 10:56, Ayub Khan wrote: > > *I'm curious about why you are using all of cloudflare and ALB and > > nginx.Seems like any one of those could provide what you are getting from > > all3 of them. * > > > > Cloudflare is doing just the DNS and nginx is doing ssl

Re: NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-11-03 Thread Ayub Khan
Christopher Schultz < ch...@christopherschultz.net> wrote: > Ayub, > > On 10/28/20 23:28, Ayub Khan wrote: > > During high load of 16k requests per minute, we notice below error in > log. > > > > [error] 2437#2437: *13335389 upstream timed out (110: Connection timed &g

NGINX + tomcat 8.0.35 (110: Connection timed out)

2020-10-28 Thread Ayub Khan
During high load of 16k requests per minute, we notice below error in log. [error] 2437#2437: *13335389 upstream timed out (110: Connection timed out) while reading response header from upstream, server: jahez.net, request: "GET /serviceContext/ServiceName?callback= HTTP/1.1", upstream: "

Re: broken pipe error keeps increasing open files

2020-06-25 Thread Ayub Khan
Chris, What do you suggest now to debug this issue ? Check with Nginx support if they can verify it ? On Thu, Jun 25, 2020 at 8:17 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ayub, > > On 6/25/2

Re: broken pipe error keeps increasing open files

2020-06-25 Thread Ayub Khan
Ayub Khan, wrote: > Chris, > > Ok, I will investigate nginx side as well. Thank you for the pointers > > On Wed, 24 Jun 2020, 19:45 Christopher Schultz, < > ch...@christopherschultz.net> wrote: > >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA256 >>

Re: broken pipe error keeps increasing open files

2020-06-24 Thread Ayub Khan
Chris, Ok, I will investigate nginx side as well. Thank you for the pointers On Wed, 24 Jun 2020, 19:45 Christopher Schultz, < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > > Ayub, > > On 6/24/20 11:05, Ayub Khan w

Re: broken pipe error keeps increasing open files

2020-06-24 Thread Ayub Khan
256 > > Ayub, > > On 6/23/20 19:17, Ayub Khan wrote: > > Yes we have nginx as reverse proxy, below is the nginx config. We > > notice this issue only when there is high number of requests, > > during non peak hours we do not see this issue.> location > > /myapp/myserv

Re: broken pipe error keeps increasing open files

2020-06-23 Thread Ayub Khan
the process. localhost:http-alt->localhost:55866 (CLOSE_WAIT) , here port 55866 is a random port. On Wed, Jun 24, 2020 at 12:48 AM Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ayub, > > On 6/23/20 16

Re: broken pipe error keeps increasing open files

2020-06-23 Thread Ayub Khan
ything related to database ip. On Mon, Jun 22, 2020 at 4:27 PM Felix Schumacher < felix.schumac...@internetallee.de> wrote: > > Am 22.06.20 um 13:22 schrieb Ayub Khan: > > Felix, > > > > I executed ls -l /proc/$(cat /var/run/tomcat8.pid)/fd/ and from the > output

Re: broken pipe error keeps increasing open files

2020-06-22 Thread Ayub Khan
] socket:[2084570167] On Mon, Jun 22, 2020 at 1:28 PM Felix Schumacher < felix.schumac...@internetallee.de> wrote: > > Am 22.06.20 um 11:41 schrieb Ayub Khan: > > Chris, > > > > I am using HikariCP for connection pooling. If the database is leaking > > conn

Re: broken pipe error keeps increasing open files

2020-06-22 Thread Ayub Khan
> Hash: SHA256 > > Ayub, > > On 6/20/20 11:51, Ayub Khan wrote: > > Sorry we are using 8.0.32 version of tomcat. > > > > below is the configuration: > > > > Server version: Apache Tomcat/8.0.32 (Ubuntu) Server built: Jan > > 24 2020 16:24:30 UTC

Re: broken pipe error keeps increasing open files

2020-06-20 Thread Ayub Khan
Calden, we are not using dbcp for this project. Also even if this error is thrown why does the file descriptor keep increasing? On Sat, Jun 20, 2020 at 8:24 PM calder wrote: > On Fri, Jun 19, 2020, 15:46 Ayub Khan wrote: > > > tomcat 8.5 broken pipe increases open files o

Re: broken pipe error keeps increasing open files

2020-06-20 Thread Ayub Khan
e from AWS load balancer. Which version of tomcat should I install ? On Sat, Jun 20, 2020 at 6:28 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ayub, > > On 6/19/20 16:46, Ayub Khan wrote: > &g

broken pipe error keeps increasing open files

2020-06-19 Thread Ayub Khan
tomcat 8.5 broken pipe increases open files on ubuntu AWS If there is slow response from db I see this stack trace and the open files goes high and the only way to open files go down is to remove the instance from Amazon load balancer. Is there a way to keep the open files low even when Broken

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-06-04 Thread Ayub Khan
gt; > On 6/4/20 12:52, Ayub Khan wrote: > > Sure I will use DNS and try to change the service calls. > > > > Also once in a while we see the below error and tomcat stops > > serving requests. Could you please let me know the cause of this > > issue ? > > &g

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-06-04 Thread Ayub Khan
: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ayub, > > On 6/4/20 11:05, Ayub Khan wrote: > > Christopher Schultz wrote: > >> There's no particular reason why a request to node1:/app1 needs > >> to have its loopback request call node1:/app2,

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-06-04 Thread Ayub Khan
) at java.lang.Thread.run(Thread.java:748) On Thu, Jun 4, 2020 at 6:05 PM Ayub Khan wrote: > Mark, > > > There's no particular reason why a request to node1:/app1 needs > to have its loopback request call node1:/app2, is there? Can > node1:/app1 call node2:/app2? > > > Yes we can do

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-06-04 Thread Ayub Khan
Hash: SHA256 > > Ayub, > > On 6/1/20 11:12, Ayub Khan wrote: > > Chris, > > > > As you described I have added two new connectors in server.xml and > > using nginx to redirect requests to different connector ports. > > Also configured nginx to route traffic o

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-06-01 Thread Ayub Khan
Christopher Schultz, wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ayub, > > On 5/31/20 09:20, Ayub Khan wrote: > > On single tomcat instance how to map each app to different port > > number? > > You'd have to use multiple elements, which mean

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-05-31 Thread Ayub Khan
Chris, On single tomcat instance how to map each app to different port number? On Sun, 31 May 2020, 15:44 Christopher Schultz, < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ayub, > > On 5/29/20 20:23, Ayub Khan wrote: >

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-05-29 Thread Ayub Khan
Ayub Khan wrote: > Chris, > > You might want (2) and (3) to have their own, independent connector > and thread pool, just to be safe. You don't want a connection in (1) > to stall because a loopback connection can't be made to (2)/(3). > Meanwhile, it's sitting there making no

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-05-29 Thread Ayub Khan
tomcat instance:* ** On Fri, May 29, 2020 at 9:05 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ayub, > > On 5/28/20 17:25, Ayub Khan wrote: > > Nginx is being used for image cachin

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-05-28 Thread Ayub Khan
instead of localhost calls ? 2) Services for portal 3) Services for portal and mobile clients On Thu, May 28, 2020 at 4:50 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ayub, > > On 5/27/20 19:43, Ay

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-05-27 Thread Ayub Khan
to handle it gracefully. On Wed, May 27, 2020 at 5:38 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ayub, > > On 5/27/20 09:26, Ayub Khan wrote: > > previously I was using HTTP/1.1 connector,

Re: tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-05-27 Thread Ayub Khan
of requests is less. On Wed, May 27, 2020 at 11:48 AM Mark Thomas wrote: > On 26/05/2020 23:28, Ayub Khan wrote: > > Hi, > > > > During high load I am seeing below error on tomcat logs > > > > java.util.concurrent.ExecutionException: java.net > .SocketTimeoutExc

tomcat 8.5 config for ecommerce site, seeing request timeouts

2020-05-26 Thread Ayub Khan
Hi, During high load I am seeing below error on tomcat logs java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: timeout We have 4 C5.18x large vms running tomcat 8 behind AWS application load balancer. We are seeing socket timeouts during peak hours. What should be the