Sorry for the noise, you gave the numbers of open files (8028) and the
limits (262144) in your first mail.

Felix

Am 23.05.21 um 12:18 schrieb Felix Schumacher:
> Am 22.05.21 um 18:03 schrieb Yeggy Javadi:
>> Here it is:
>>
>> # netstat -p -a  --tcp | grep 130244
>> tcp6       0      0 [::]:pcsync-https       [::]:*                  LISTEN   
>>    130244/java
>> tcp6       0      0 [::]:https              [::]:*                  LISTEN   
>>    130244/java
>> tcp6       0      0 [::]:37537              [::]:*                  LISTEN   
>>    130244/java
>> tcp6       0      0 localhost.localdoma:mxi [::]:*                  LISTEN   
>>    130244/java
>> tcp6       0      0 [::]:8009               [::]:*                  LISTEN   
>>    130244/java
>> tcp6       0      0 [::]:rmiregistry        [::]:*                  LISTEN   
>>    130244/java
>> tcp6       0      0 [::]:http               [::]:*                  LISTEN   
>>    130244/java
>> tcp6      86      0 Yeggy-F8-FMSVA:39680    172.22.22.192:https     
>> CLOSE_WAIT  130244/java
>> tcp6       0      1 Yeggy-F8-FMSVA:53618    10.12.3.78:https        SYN_SENT 
>>    130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54772    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 localhost.localdo:42664 localhost.loca:postgres 
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54782    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54766    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 localhost.localdo:42662 localhost.loca:postgres 
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54778    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54788    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54770    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54790    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54776    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54786    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54780    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 localhost.localdo:45736 localhost.loca:postgres 
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54768    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54784    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 localhost.localdo:42660 localhost.loca:postgres 
>> ESTABLISHED 130244/java
>> tcp6       0      1 Yeggy-F8-FMSVA:42922    10.12.3.77:https        SYN_SENT 
>>    130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:35794    172.22.22.192:https     
>> ESTABLISHED 130244/java
>> tcp6       0      0 Yeggy-F8-FMSVA:54774    Yeggy-F8-FMSVA:vrace    
>> ESTABLISHED 130244/java
>> tcp6       0      0 localhost.localdo:45734 localhost.loca:postgres 
>> ESTABLISHED 130244/java
>> tcp6       0      0 localhost.localdo:41016 localhost.localdo:vrace 
>> ESTABLISHED 130244/java
>>
>>
>> # lsof -p 130244
>> COMMAND    PID USER   FD      TYPE             DEVICE SIZE/OFF     NODE NAME
>> java    130244 root  cwd       DIR                8,2     4096   157664 
>> /usr/local/freestor/bin
>> java    130244 root  rtd       DIR                8,3     4096        2 /
>> java    130244 root  txt       REG                8,2     8712     8913 
>> /usr/local/jdk/jre1.8.0_271/bin/java
>> java    130244 root  mem       REG                8,2   498864     9007 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libfontmanager.so
>> java    130244 root  mem       REG                8,2    39176     9006 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libawt_headless.so
>> java    130244 root  mem       REG                8,2   759184     8996 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libawt.so
>> java    130244 root  mem       REG                8,2  3559360     9139 
>> /usr/local/jdk/jre1.8.0_271/lib/resources.jar
>> java    130244 root  mem       REG                8,2    99680   133076 
>> /usr/lib64/libgcc_s-8-20191121.so.1
>> java    130244 root  mem       REG                8,2  3135658     9133 
>> /usr/local/jdk/jre1.8.0_271/lib/charsets.jar
>> java    130244 root  mem       REG                8,2   283368     8980 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libsunec.so
>> java    130244 root  mem       REG                8,2  1907341     9138 
>> /usr/local/jdk/jre1.8.0_271/lib/jsse.jar
>> java    130244 root  mem       REG                8,2   285868     9126 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/sunjce_provider.jar
>> java    130244 root  mem       REG                8,2    43191     9125 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/sunec.jar
>> java    130244 root  mem       REG                8,2   283206     9127 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/sunpkcs11.jar
>> java    130244 root  mem       REG                8,2   118047     9136 
>> /usr/local/jdk/jre1.8.0_271/lib/jce.jar
>> java    130244 root  mem       REG                8,2  1179462     9128 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/localedata.jar
>> java    130244 root  mem       REG                8,2  3860563     9129 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/cldrdata.jar
>> java    130244 root  mem       REG                8,2    51984     8995 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libmanagement.so
>> java    130244 root  mem       REG                8,2    12373     8871 
>> /usr/local/apache-tomcat-8.5.59/lib/annotations-api.jar
>> java    130244 root  mem       REG                8,2   169952     8868 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-util.jar
>> java    130244 root  mem       REG                8,2   168382     8867 
>> /usr/local/apache-tomcat-8.5.59/lib/jasper-el.jar
>> java    130244 root  mem       REG                8,2   244451     8862 
>> /usr/local/apache-tomcat-8.5.59/lib/servlet-api.jar
>> java    130244 root  mem       REG                8,2   148761     8881 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-jdbc.jar
>> java    130244 root  mem       REG                8,2    62004     8863 
>> /usr/local/apache-tomcat-8.5.59/lib/jsp-api.jar
>> java    130244 root  mem       REG                8,2  2450404     8864 
>> /usr/local/apache-tomcat-8.5.59/lib/ecj-4.6.3.jar
>> java    130244 root  mem       REG                8,2    34798     8873 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-jni.jar
>> java    130244 root  mem       REG                8,2    78087     8886 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina-storeconfig.jar
>> java    130244 root  mem       REG                8,2    89695     8877 
>> /usr/local/apache-tomcat-8.5.59/lib/el-api.jar
>> java    130244 root  mem       REG                8,2   599431     8884 
>> /usr/local/apache-tomcat-8.5.59/lib/jasper.jar
>> java    130244 root  mem       REG                8,2   174393     8870 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-ja.jar
>> java    130244 root  mem       REG                8,2   294856     8861 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina-tribes.jar
>> java    130244 root  mem       REG                8,2   173282     8880 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-ko.jar
>> java    130244 root  mem       REG                8,2   284036     8872 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-dbcp.jar
>> java    130244 root  mem       REG                8,2    72656     8865 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-de.jar
>> java    130244 root  mem       REG                8,2    54667     8878 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina-ant.jar
>> java    130244 root  mem       REG                8,2  1705863     8860 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina.jar
>> java    130244 root  mem       REG                8,2    38241     8882 
>> /usr/local/apache-tomcat-8.5.59/lib/websocket-api.jar
>> java    130244 root  mem       REG                8,2   106051     8869 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-es.jar
>> java    130244 root  mem       REG                8,2   120073     8876 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina-ha.jar
>> java    130244 root  mem       REG                8,2   228391     8879 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-websocket.jar
>> java    130244 root  mem       REG                8,2   868336     8888 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-coyote.jar
>> java    130244 root  mem       REG                8,2   215400     8874 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-util-scan.jar
>> java    130244 root  mem       REG                8,2   155094     8866 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-fr.jar
>> java    130244 root  mem       REG                8,2    26872     8885 
>> /usr/local/apache-tomcat-8.5.59/lib/jaspic-api.jar
>> java    130244 root  mem       REG                8,2   113008     8989 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libnet.so
>> java    130244 root  mem       REG                8,2    93872     9003 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libnio.so
>> java    130244 root  mem       REG                8,2   337024   395971 
>> /usr/lib/locale/en_US.utf8/LC_CTYPE
>> java    130244 root  mem       REG                8,2  2586930   390297 
>> /usr/lib/locale/en_US.utf8/LC_COLLATE
>> java    130244 root  mem       REG                8,2    10673     8887 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-api.jar
>> java    130244 root  mem       REG                8,2    38561     8875 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-ru.jar
>> java    130244 root  mem       REG                8,2   158590     8883 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-zh-CN.jar
>> java    130244 root  mem       REG                8,2    51205     8857 
>> /usr/local/apache-tomcat-8.5.59/bin/tomcat-juli.jar
>> java    130244 root  mem       REG                8,2    25287     8835 
>> /usr/local/apache-tomcat-8.5.59/bin/commons-daemon.jar
>> java    130244 root  mem       REG                8,2    36132     8837 
>> /usr/local/apache-tomcat-8.5.59/bin/bootstrap.jar
>> java    130244 root  mem       REG                8,2       54   395975 
>> /usr/lib/locale/en_US.utf8/LC_NUMERIC
>> java    130244 root  mem       REG                8,2     3316   521695 
>> /usr/lib/locale/en_US.utf8/LC_TIME
>> java    130244 root  mem       REG                8,2      286   521693 
>> /usr/lib/locale/en_US.utf8/LC_MONETARY
>> java    130244 root  mem       REG                8,2    26998   130155 
>> /usr/lib64/gconv/gconv-modules.cache
>> java    130244 root  mem       REG                8,2 67053209     9140 
>> /usr/local/jdk/jre1.8.0_271/lib/rt.jar
>> java    130244 root  mem       REG                8,2   127016     8988 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libzip.so
>> java    130244 root  mem       REG                8,2    76880   133068 
>> /usr/lib64/libnss_files-2.28.so
>> java    130244 root  mem       REG                8,3  9253600   129922 
>> /var/lib/sss/mc/passwd
>> java    130244 root  mem       REG                8,2    46432   134002 
>> /usr/lib64/libnss_sss.so.2
>> java    130244 root  mem       REG                8,2   231960     8986 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libjava.so
>> java    130244 root  mem       REG                8,2    66112     8975 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/libverify.so
>> java    130244 root  mem       REG                8,2    68944   133091 
>> /usr/lib64/librt-2.28.so
>> java    130244 root  mem       REG                8,2  2191808   133023 
>> /usr/lib64/libm-2.28.so
>> java    130244 root  mem       REG                8,2 17108824     8969 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/server/libjvm.so
>> java    130244 root  mem       REG                8,2  3154704   129871 
>> /usr/lib64/libc-2.28.so
>> java    130244 root  mem       REG                8,2    28968   133013 
>> /usr/lib64/libdl-2.28.so
>> java    130244 root  mem       REG                8,2   108616     8973 
>> /usr/local/jdk/jre1.8.0_271/lib/amd64/jli/libjli.so
>> java    130244 root  mem       REG                8,2   321552   133079 
>> /usr/lib64/libpthread-2.28.so
>> java    130244 root  mem       REG                8,2   252280   129864 
>> /usr/lib64/ld-2.28.so
>> java    130244 root  mem       REG                8,3    32768       18 
>> /tmp/hsperfdata_root/130244
>> java    130244 root  mem       REG                8,2       57   395973 
>> /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES
>> java    130244 root  mem       REG                8,2       34   390317 
>> /usr/lib/locale/en_US.utf8/LC_PAPER
>> java    130244 root  mem       REG                8,2       77   395974 
>> /usr/lib/locale/en_US.utf8/LC_NAME
>> java    130244 root  mem       REG                8,2      167   519390 
>> /usr/lib/locale/en_US.utf8/LC_ADDRESS
>> java    130244 root  mem       REG                8,2       59   521694 
>> /usr/lib/locale/en_US.utf8/LC_TELEPHONE
>> java    130244 root  mem       REG                8,2       23   521692 
>> /usr/lib/locale/en_US.utf8/LC_MEASUREMENT
>> java    130244 root  mem       REG                8,2      368   521691 
>> /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION
>> java    130244 root    0r      CHR                1,3      0t0     1037 
>> /dev/null
>> java    130244 root    1w      REG                8,6   394654  1835011 
>> /apps/apache-tomcat/logs/catalina.out
>> java    130244 root    2w      REG                8,6   394654  1835011 
>> /apps/apache-tomcat/logs/catalina.out
>> java    130244 root    3r      REG                8,3  9253600   129922 
>> /var/lib/sss/mc/passwd
>> java    130244 root    4u     unix 0xffff985d1bd7b180      0t0 11980864 
>> type=STREAM
>> java    130244 root    5r      REG                8,2 67053209     9140 
>> /usr/local/jdk/jre1.8.0_271/lib/rt.jar
>> java    130244 root    6r      REG                8,2    36132     8837 
>> /usr/local/apache-tomcat-8.5.59/bin/bootstrap.jar
>> java    130244 root    7r      REG                8,2    25287     8835 
>> /usr/local/apache-tomcat-8.5.59/bin/commons-daemon.jar
>> java    130244 root    8r      REG                8,2    51205     8857 
>> /usr/local/apache-tomcat-8.5.59/bin/tomcat-juli.jar
>> java    130244 root    9w      REG                8,6    21883  1835065 
>> /apps/apache-tomcat/logs/catalina.2021-05-20.log
>> java    130244 root   10w      REG                8,6      761  1835066 
>> /apps/apache-tomcat/logs/localhost.2021-05-20.log
>> java    130244 root   11w      REG                8,6        0  1835067 
>> /apps/apache-tomcat/logs/manager.2021-05-20.log
>> java    130244 root   12w      REG                8,6        0  1835068 
>> /apps/apache-tomcat/logs/host-manager.2021-05-20.log
>> java    130244 root   13r      REG                8,2   158590     8883 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-zh-CN.jar
>> java    130244 root   14r      REG                8,2    38561     8875 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-ru.jar
>> java    130244 root   15r      REG                8,2    26872     8885 
>> /usr/local/apache-tomcat-8.5.59/lib/jaspic-api.jar
>> java    130244 root   16r      REG                8,2   155094     8866 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-fr.jar
>> java    130244 root   17r      REG                8,2   215400     8874 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-util-scan.jar
>> java    130244 root   18r      REG                8,2   868336     8888 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-coyote.jar
>> java    130244 root   19r      REG                8,2   228391     8879 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-websocket.jar
>> java    130244 root   20r      REG                8,2   120073     8876 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina-ha.jar
>> java    130244 root   21r      REG                8,2   106051     8869 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-es.jar
>> java    130244 root   22r      REG                8,2    38241     8882 
>> /usr/local/apache-tomcat-8.5.59/lib/websocket-api.jar
>> java    130244 root   23r      REG                8,2  1705863     8860 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina.jar
>> java    130244 root   24r      REG                8,2    54667     8878 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina-ant.jar
>> java    130244 root   25r      REG                8,2    72656     8865 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-de.jar
>> java    130244 root   26r      REG                8,2   284036     8872 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-dbcp.jar
>> java    130244 root   27r      REG                8,2    10673     8887 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-api.jar
>> java    130244 root   28r      REG                8,2   173282     8880 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-ko.jar
>> java    130244 root   29r      REG                8,2   294856     8861 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina-tribes.jar
>> java    130244 root   30r      REG                8,2   174393     8870 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-i18n-ja.jar
>> java    130244 root   31r      REG                8,2   599431     8884 
>> /usr/local/apache-tomcat-8.5.59/lib/jasper.jar
>> java    130244 root   32r      REG                8,2    89695     8877 
>> /usr/local/apache-tomcat-8.5.59/lib/el-api.jar
>> java    130244 root   33r      REG                8,2    78087     8886 
>> /usr/local/apache-tomcat-8.5.59/lib/catalina-storeconfig.jar
>> java    130244 root   34r      REG                8,2    34798     8873 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-jni.jar
>> java    130244 root   35r      REG                8,2  2450404     8864 
>> /usr/local/apache-tomcat-8.5.59/lib/ecj-4.6.3.jar
>> java    130244 root   36r      REG                8,2    62004     8863 
>> /usr/local/apache-tomcat-8.5.59/lib/jsp-api.jar
>> java    130244 root   37r      REG                8,2   148761     8881 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-jdbc.jar
>> java    130244 root   38r      REG                8,2   244451     8862 
>> /usr/local/apache-tomcat-8.5.59/lib/servlet-api.jar
>> java    130244 root   39r      REG                8,2   168382     8867 
>> /usr/local/apache-tomcat-8.5.59/lib/jasper-el.jar
>> java    130244 root   40r      REG                8,2   169952     8868 
>> /usr/local/apache-tomcat-8.5.59/lib/tomcat-util.jar
>> java    130244 root   41r      REG                8,2    12373     8871 
>> /usr/local/apache-tomcat-8.5.59/lib/annotations-api.jar
>> java    130244 root   42r      REG                8,2  1907341     9138 
>> /usr/local/jdk/jre1.8.0_271/lib/jsse.jar
>> java    130244 root   43r      REG                8,2  3860563     9129 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/cldrdata.jar
>> java    130244 root   44u     unix 0xffff985d1bd7a400      0t0 11980871 
>> type=STREAM
>> java    130244 root   45r      REG                8,2  1179462     9128 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/localedata.jar
>> java    130244 root   46r      CHR                1,8      0t0     1041 
>> /dev/random
>> java    130244 root   47r      CHR                1,9      0t0     1042 
>> /dev/urandom
>> java    130244 root   48r      CHR                1,8      0t0     1041 
>> /dev/random
>> java    130244 root   49r      CHR                1,8      0t0     1041 
>> /dev/random
>> java    130244 root   50r      CHR                1,9      0t0     1042 
>> /dev/urandom
>> java    130244 root   51r      CHR                1,9      0t0     1042 
>> /dev/urandom
>> java    130244 root   52r      REG                8,2   118047     9136 
>> /usr/local/jdk/jre1.8.0_271/lib/jce.jar
>> java    130244 root   53r      REG                8,2   283206     9127 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/sunpkcs11.jar
>> java    130244 root   54r      REG                8,2    43191     9125 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/sunec.jar
>> java    130244 root   55r      REG                8,2   285868     9126 
>> /usr/local/jdk/jre1.8.0_271/lib/ext/sunjce_provider.jar
>> java    130244 root   56r      REG                8,2  3559360     9139 
>> /usr/local/jdk/jre1.8.0_271/lib/resources.jar
>> java    130244 root   57r      REG                8,2  3135658     9133 
>> /usr/local/jdk/jre1.8.0_271/lib/charsets.jar
>> java    130244 root   58u     IPv6           11980876      0t0      TCP 
>> *:http (LISTEN)
>> java    130244 root   59u     unix 0xffff985d1bd7e780      0t0 11980877 
>> type=STREAM
>> java    130244 root   60r     FIFO               0,13      0t0 11980879 pipe
>> java    130244 root   61w     FIFO               0,13      0t0 11980879 pipe
>> java    130244 root   62u  a_inode               0,14        0    10970 
>> [eventpoll]
>> java    130244 root   63u     IPv6           11980880      0t0      TCP 
>> *:https (LISTEN)
>> java    130244 root   64r     FIFO               0,13      0t0 11980884 pipe
>> java    130244 root   65w     FIFO               0,13      0t0 11980884 pipe
>> java    130244 root   66u  a_inode               0,14        0    10970 
>> [eventpoll]
>> java    130244 root   67u     IPv6           11980885      0t0      TCP 
>> *:8009 (LISTEN)
>> java    130244 root   68r     FIFO               0,13      0t0 11980886 pipe
>> java    130244 root   69w     FIFO               0,13      0t0 11980886 pipe
>> java    130244 root   70u  a_inode               0,14        0    10970 
>> [eventpoll]
>> java    130244 root   71w      REG                8,6   342714  1835069 
>> /apps/apache-tomcat/logs/localhost_access_log.2021-05-21.txt
>> java    130244 root   72r     FIFO               0,13      0t0 11968845 pipe
>> java    130244 root   73w     FIFO               0,13      0t0 11968845 pipe
>> java    130244 root   74u  a_inode               0,14        0    10970 
>> [eventpoll]
>> java    130244 root   75r     FIFO               0,13      0t0 11968846 pipe
>> java    130244 root   76w     FIFO               0,13      0t0 11968846 pipe
>> java    130244 root   77u  a_inode               0,14        0    10970 
>> [eventpoll]
>> java    130244 root   78u     sock                0,9      0t0 12154374 
>> protocol: TCPv6
>> java    130244 root   79r     FIFO               0,13      0t0 11968847 pipe
>> java    130244 root   80w     FIFO               0,13      0t0 11968847 pipe
>> java    130244 root   81u  a_inode               0,14        0    10970 
>> [eventpoll]
>> java    130244 root   82r     FIFO               0,13      0t0 11968848 pipe
>> java    130244 root   83w     FIFO               0,13      0t0 11968848 pipe
>> java    130244 root   84u  a_inode               0,14        0    10970 
>> [eventpoll]
>> java    130244 root   86r     FIFO               0,13      0t0 11968849 pipe
>> java    130244 root   87w     FIFO               0,13      0t0 11968849 pipe
>> java    130244 root   88u  a_inode               0,14        0    10970 
>> [eventpoll]
>> java    130244 root   89r     FIFO               0,13      0t0 11968850 pipe
>> java    130244 root   90w     FIFO               0,13      0t0 11968850 pipe
>> java    130244 root   91u  a_inode               0,14        0    10970 
>> [eventpoll]
>> java    130244 root   93u     IPv6           11968853      0t0      TCP 
>> localhost.localdomain:mxi (LISTEN)
>> java    130244 root   94u     IPv6           12298916      0t0      TCP 
>> localhost.localdomain:45734->localhost.localdomain:postgres (ESTABLISHED)
>> java    130244 root   95u     sock                0,9      0t0 12017983 
>> protocol: TCPv6
>> java    130244 root   96u     sock                0,9      0t0 11983086 
>> protocol: TCPv6
>> java    130244 root   97u     sock                0,9      0t0 11983087 
>> protocol: TCPv6
>> java    130244 root   98u     sock                0,9      0t0 11983088 
>> protocol: TCPv6
>> java    130244 root   99u     sock                0,9      0t0 11983089 
>> protocol: TCPv6
>> java    130244 root  100u     sock                0,9      0t0 11983090 
>> protocol: TCPv6
>> java    130244 root  101u     sock                0,9      0t0 11983091 
>> protocol: TCPv6
>> java    130244 root  102u     sock                0,9      0t0 11983092 
>> protocol: TCPv6
>> java    130244 root  103u     sock                0,9      0t0 11969375 
>> protocol: TCPv6
>> java    130244 root  104u     sock                0,9      0t0 11987339 
>> protocol: TCPv6
>> java    130244 root  105u     sock                0,9      0t0 11987347 
>> protocol: TCPv6
>> java    130244 root  106u     sock                0,9      0t0 11969384 
>> protocol: TCPv6
>> java    130244 root  107u     sock                0,9      0t0 11987348 
>> protocol: TCPv6
>> java    130244 root  108r      CHR                1,8      0t0     1041 
>> /dev/random
>> java    130244 root  109w      REG                8,6 19536380  1835053 
>> /apps/apache-tomcat/logs/fms.log
>> java    130244 root  110u     sock                0,9      0t0 11979291 
>> protocol: TCPv6
>> java    130244 root  111u     sock                0,9      0t0 11979292 
>> protocol: TCPv6
>> java    130244 root  112w      REG                8,6   835408  1835019 
>> /apps/apache-tomcat/logs/obd.log
>> java    130244 root  113u     sock                0,9      0t0 11987349 
>> protocol: TCPv6
>> java    130244 root  114u     sock                0,9      0t0 11987350 
>> protocol: TCPv6
>> java    130244 root  115u     sock                0,9      0t0 11987351 
>> protocol: TCPv6
>> java    130244 root  116u     sock                0,9      0t0 11987352 
>> protocol: TCPv6
>> java    130244 root  117u     sock                0,9      0t0 11987353 
>> protocol: TCPv6
>> java    130244 root  118u     sock                0,9      0t0 11987354 
>> protocol: TCPv6
>> java    130244 root  119u     sock                0,9      0t0 11987355 
>> protocol: TCPv6
>> java    130244 root  120u     sock                0,9      0t0 11987356 
>> protocol: TCPv6
>> java    130244 root  121u     sock                0,9      0t0 11987357 
>> protocol: TCPv6
>> java    130244 root  122u     sock                0,9      0t0 11987358 
>> protocol: TCPv6
>> java    130244 root  123u     sock                0,9      0t0 11987359 
>> protocol: TCPv6
>> java    130244 root  124u     sock                0,9      0t0 11987360 
>> protocol: TCPv6
>> ...........................
>> Keep seeing the same line
>
> How many of those lines are there? (lsof -p ... | wc -l)
>
> Are you sure there are more than before? Maybe your ulimits where higher
> on your older system and are now
> bothering you?
>
> What is the setting of ulimit for Tomcats process? (cat /proc/<PID>/limits)
>
> Felix
>
>
>> -----Original Message-----
>> From: André Warnier (tomcat/perl) <a...@ice-sa.com> 
>> Sent: Saturday, May 22, 2021 10:22 AM
>> To: users@tomcat.apache.org
>> Subject: Re: apache-tomcat-8.5.59 too many open files on Linux 8
>>
>> Mmm. Nothing very special in that netstat output.
>> The sockets seen there look quite normal for tomcat, and there are not a lot.
>> What about the IPv4 sockets ? (remove the -6 in your netstat command)
>>
>> Looks like lsof is counting things which are not IPv6 TCP sockets belonging 
>> to the tomcat JVM process.
>> Where is the difference between the lsof count (19948) and the netstat count 
>> (25) ?
>> (genuine question, I'm puzzled too)
>>
>> Can you give an example of the "lsof -p 130244" output lines ?
>> (not all 19948 please, but enough to see some variety)
>>
>> On 21.05.2021 16:11, Yeggy Javadi wrote:
>>> Hi,
>>> Here its is:
>>> # lsof -p 130244 | grep "protocol: TCPv6"| wc -l
>>> 19948
>>>
>>> # netstat -p -a -6 --tcp | grep 130244
>>> tcp6       0      0 [::]:pcsync-https       [::]:*                  LISTEN  
>>>     130244/java
>>> tcp6       0      0 [::]:https              [::]:*                  LISTEN  
>>>     130244/java
>>> tcp6       0      0 [::]:37537              [::]:*                  LISTEN  
>>>     130244/java
>>> tcp6       0      0 localhost.localdoma:mxi [::]:*                  LISTEN  
>>>     130244/java
>>> tcp6       0      0 [::]:8009               [::]:*                  LISTEN  
>>>     130244/java
>>> tcp6       0      0 [::]:rmiregistry        [::]:*                  LISTEN  
>>>     130244/java
>>> tcp6       0      0 [::]:http               [::]:*                  LISTEN  
>>>     130244/java
>>> tcp6      86      0 Yeggy-F8-FMSVA:39680    172.22.22.192:https     
>>> CLOSE_WAIT  130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54772    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 localhost.localdo:42664 localhost.loca:postgres 
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54782    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54766    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 localhost.localdo:42662 localhost.loca:postgres 
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54778    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 localhost.localdo:46966 localhost.localdo:11753 
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54788    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54770    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:http     10.197.255.10:64799     
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54790    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54776    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54786    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54780    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 localhost.localdo:45736 localhost.loca:postgres 
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54768    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54784    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 localhost.localdo:42660 localhost.loca:postgres 
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:54774    Yeggy-F8-FMSVA:vrace    
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 localhost.localdo:45734 localhost.loca:postgres 
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 Yeggy-F8-FMSVA:http     10.197.255.10:64798     
>>> ESTABLISHED 130244/java
>>> tcp6       0      0 localhost.localdo:41016 localhost.localdo:vrace 
>>> ESTABLISHED 130244/java
>>> Thanks
>>> -----Original Message-----
>>> From: Noelette Stout <noelettest...@isu.edu>
>>> Sent: Friday, May 21, 2021 8:28 AM
>>> To: Tomcat Users List <users@tomcat.apache.org>
>>> Subject: Re: apache-tomcat-8.5.59 too many open files on Linux 8
>>>
>>> ss has all the same options as netstat
>>>
>>> On Fri, May 21, 2021 at 3:51 AM André Warnier (tomcat/perl) 
>>> <a...@ice-sa.com>
>>> wrote:
>>>
>>>> Soyrry to top-post, but it's getting cluttered down there..
>>>>
>>>> The next thing that you may want to do :
>>>>
>>>>    > netstat -p -a -6 --tcp
>>>>
>>>> That is an alternative list of sockets, which also shows the "tcp state"
>>>> of the sockets.
>>>> To get only the ones of the tomcat JVM PID, filter with grep based of 
>>>> the last column.
>>>> The type of thing you are looking for is the column which should show 
>>>> "LISTEN", or "ESTABLISHED" or "CLOSE_WAIT" etc..
>>>>
>>>> The options above :
>>>> -p : show PID and program
>>>> -a : show all sockets states
>>>> -6 : only inet v6
>>>> --tcp : only TCP sockets
>>>>
>>>> "netstat" may not be on your system by default, and you may need to 
>>>> install it.
>>>> An alternative is "ss", but I don't know the options.
>>>>
>>>>
>>>> On 21.05.2021 02:14, Yeggy Javadi wrote:
>>>>> Hi,
>>>>> Yes; that is what I get and as you can see among 8028 open files,
>>>>> 7474
>>>> are for TCPv6 sockets:
>>>>> java    130244 root 7805u     sock                0,9      0t0 12294251
>>>> protocol: TCPv6
>>>>> # ps -ef | grep tomcat
>>>>> root      130244       1  1 11:01 ?        00:06:20
>>>> /usr/local/jre/bin/java
>>>> -Djava.util.logging.config.file=/usr/local/apache-tomcat/conf/logging.
>>>> properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 
>>>> -d64 -server -Xms1800m -Xmx8192m -XX:MaxMetaspaceSize=1800m
>>>> -Djdk.tls.ephemeralDHKeySize=2048
>>>> -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
>>>> -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
>>>> -Dignore.endorsed.dirs= -classpath
>>>> /usr/local/apache-tomcat/bin/bootstrap.jar:/usr/local/apache-tomcat/b
>>>> i n/tomcat-juli.jar -Dcatalina.base=/usr/local/apache-tomcat
>>>> -Dcatalina.home=/usr/local/apache-tomcat
>>>> -Djava.io.tmpdir=/usr/local/apache-tomcat/temp
>>>> org.apache.catalina.startup.Bootstrap start
>>>>> root      132566  132492  0 20:10 pts/1    00:00:00 grep --color=auto
>>>> tomcat
>>>>> #lsof -p 130244 | wc -l
>>>>> 8028
>>>>>
>>>>> #lsof -p 130244 | grep "protocol: TCPv6"| wc -l
>>>>> 7474
>>>>>
>>>>> Thanks
>>>>>
>>>>> -----Original Message-----
>>>>> From: André Warnier (tomcat/perl) <a...@ice-sa.com>
>>>>> Sent: Thursday, May 20, 2021 4:19 PM
>>>>> To: users@tomcat.apache.org
>>>>> Subject: Re: apache-tomcat-8.5.59 too many open files on Linux 8
>>>>>
>>>>> Hi.
>>>>> According to the list below, you have 2 java (JVM) processes running 
>>>>> on
>>>> your system.
>>>>> One (PID = 130244) is the JVM which runs tomcat.  This is visible 
>>>>> when
>>>> you look at the whole command-line.
>>>>> The other (PID = 130516) runs ElasticSearch, which I believe is not
>>>> relevant here.
>>>>> So you should run lsof with the "-p 130244" option, to show only the
>>>> files opened by the tomcat JVM.
>>>>> To show the current Tomcat JVM PID, do e.g. : ps -ef | grep
>>>> "apache-tomcat"
>>>>> (or anything else unique in that line)
>>>>>
>>>>>
>>>>> On 20.05.2021 21:00, Yeggy Javadi wrote:
>>>>>> Hi Chris,
>>>>>> Please indicate how to show lsof or netstat to *just the JVM* process.
>>>>>>
>>>>>> Below is the list of running processes on my server:
>>>>>> UID          PID    PPID  C STIME TTY          TIME CMD
>>>>>> root           1       0  0 May07 ?        00:00:14
>>>> /usr/lib/systemd/systemd --switched-root --system --deserialize 17
>>>>>> root           2       0  0 May07 ?        00:00:00 [kthreadd]
>>>>>> root           3       2  0 May07 ?        00:00:00 [rcu_gp]
>>>>>> root           4       2  0 May07 ?        00:00:00 [rcu_par_gp]
>>>>>> root           6       2  0 May07 ?        00:00:00
>>>> [kworker/0:0H-kblockd]
>>>>>> root           8       2  0 May07 ?        00:00:00 [mm_percpu_wq]
>>>>>> root           9       2  0 May07 ?        00:00:00 [ksoftirqd/0]
>>>>>> root          10       2  0 May07 ?        00:02:18 [rcu_sched]
>>>>>> root          11       2  0 May07 ?        00:00:00 [migration/0]
>>>>>> root          12       2  0 May07 ?        00:00:00 [watchdog/0]
>>>>>> root          13       2  0 May07 ?        00:00:00 [cpuhp/0]
>>>>>> root          14       2  0 May07 ?        00:00:00 [cpuhp/1]
>>>>>> root          15       2  0 May07 ?        00:00:00 [watchdog/1]
>>>>>> root          16       2  0 May07 ?        00:00:00 [migration/1]
>>>>>> root          17       2  0 May07 ?        00:00:00 [ksoftirqd/1]
>>>>>> root          19       2  0 May07 ?        00:00:00
>>>> [kworker/1:0H-kblockd]
>>>>>> root          20       2  0 May07 ?        00:00:00 [cpuhp/2]
>>>>>> root          21       2  0 May07 ?        00:00:00 [watchdog/2]
>>>>>> root          22       2  0 May07 ?        00:00:00 [migration/2]
>>>>>> root          23       2  0 May07 ?        00:00:00 [ksoftirqd/2]
>>>>>> root          25       2  0 May07 ?        00:00:00
>>>> [kworker/2:0H-kblockd]
>>>>>> root          26       2  0 May07 ?        00:00:00 [cpuhp/3]
>>>>>> root          27       2  0 May07 ?        00:00:00 [watchdog/3]
>>>>>> root          28       2  0 May07 ?        00:00:00 [migration/3]
>>>>>> root          29       2  0 May07 ?        00:00:00 [ksoftirqd/3]
>>>>>> root          31       2  0 May07 ?        00:00:00
>>>> [kworker/3:0H-kblockd]
>>>>>> root          32       2  0 May07 ?        00:00:00 [cpuhp/4]
>>>>>> root          33       2  0 May07 ?        00:00:00 [watchdog/4]
>>>>>> root          34       2  0 May07 ?        00:00:00 [migration/4]
>>>>>> root          35       2  0 May07 ?        00:00:00 [ksoftirqd/4]
>>>>>> root          37       2  0 May07 ?        00:00:00
>>>> [kworker/4:0H-kblockd]
>>>>>> root          38       2  0 May07 ?        00:00:00 [cpuhp/5]
>>>>>> root          39       2  0 May07 ?        00:00:00 [watchdog/5]
>>>>>> root          40       2  0 May07 ?        00:00:00 [migration/5]
>>>>>> root          41       2  0 May07 ?        00:00:00 [ksoftirqd/5]
>>>>>> root          43       2  0 May07 ?        00:00:00
>>>> [kworker/5:0H-kblockd]
>>>>>> root          44       2  0 May07 ?        00:00:00 [cpuhp/6]
>>>>>> root          45       2  0 May07 ?        00:00:00 [watchdog/6]
>>>>>> root          46       2  0 May07 ?        00:00:00 [migration/6]
>>>>>> root          47       2  0 May07 ?        00:00:00 [ksoftirqd/6]
>>>>>> root          49       2  0 May07 ?        00:00:00
>>>> [kworker/6:0H-kblockd]
>>>>>> root          50       2  0 May07 ?        00:00:00 [cpuhp/7]
>>>>>> root          51       2  0 May07 ?        00:00:00 [watchdog/7]
>>>>>> root          52       2  0 May07 ?        00:00:00 [migration/7]
>>>>>> root          53       2  0 May07 ?        00:00:00 [ksoftirqd/7]
>>>>>> root          55       2  0 May07 ?        00:00:00
>>>> [kworker/7:0H-kblockd]
>>>>>> root          57       2  0 May07 ?        00:00:00 [kdevtmpfs]
>>>>>> root          58       2  0 May07 ?        00:00:00 [netns]
>>>>>> root          59       2  0 May07 ?        00:00:00 [kauditd]
>>>>>> root          62       2  0 May07 ?        00:00:00 [khungtaskd]
>>>>>> root          63       2  0 May07 ?        00:00:00 [oom_reaper]
>>>>>> root          64       2  0 May07 ?        00:00:00 [writeback]
>>>>>> root          65       2  0 May07 ?        00:00:00 [kcompactd0]
>>>>>> root          66       2  0 May07 ?        00:00:00 [ksmd]
>>>>>> root          67       2  0 May07 ?        00:00:02 [khugepaged]
>>>>>> root          68       2  0 May07 ?        00:00:00 [crypto]
>>>>>> root          69       2  0 May07 ?        00:00:00 [kintegrityd]
>>>>>> root          70       2  0 May07 ?        00:00:00 [kblockd]
>>>>>> root          71       2  0 May07 ?        00:00:00 [tpm_dev_wq]
>>>>>> root          72       2  0 May07 ?        00:00:00 [md]
>>>>>> root          73       2  0 May07 ?        00:00:00 [edac-poller]
>>>>>> root          74       2  0 May07 ?        00:00:00 [watchdogd]
>>>>>> root          92       2  0 May07 ?        00:00:00 [kswapd0]
>>>>>> root         188       2  0 May07 ?        00:00:00 [kthrotld]
>>>>>> root         189       2  0 May07 ?        00:00:00 [irq/24-pciehp]
>>>>>> root         190       2  0 May07 ?        00:00:00 [irq/25-pciehp]
>>>>>> root         191       2  0 May07 ?        00:00:00 [irq/26-pciehp]
>>>>>> root         192       2  0 May07 ?        00:00:00 [irq/27-pciehp]
>>>>>> root         193       2  0 May07 ?        00:00:00 [irq/28-pciehp]
>>>>>> root         194       2  0 May07 ?        00:00:00 [irq/29-pciehp]
>>>>>> root         195       2  0 May07 ?        00:00:00 [irq/30-pciehp]
>>>>>> root         196       2  0 May07 ?        00:00:00 [irq/31-pciehp]
>>>>>> root         197       2  0 May07 ?        00:00:00 [irq/32-pciehp]
>>>>>> root         198       2  0 May07 ?        00:00:00 [irq/33-pciehp]
>>>>>> root         199       2  0 May07 ?        00:00:00 [irq/34-pciehp]
>>>>>> root         200       2  0 May07 ?        00:00:00 [irq/35-pciehp]
>>>>>> root         201       2  0 May07 ?        00:00:00 [irq/36-pciehp]
>>>>>> root         202       2  0 May07 ?        00:00:00 [irq/37-pciehp]
>>>>>> root         203       2  0 May07 ?        00:00:00 [irq/38-pciehp]
>>>>>> root         204       2  0 May07 ?        00:00:00 [irq/39-pciehp]
>>>>>> root         205       2  0 May07 ?        00:00:00 [irq/40-pciehp]
>>>>>> root         206       2  0 May07 ?        00:00:00 [irq/41-pciehp]
>>>>>> root         207       2  0 May07 ?        00:00:00 [irq/42-pciehp]
>>>>>> root         208       2  0 May07 ?        00:00:00 [irq/43-pciehp]
>>>>>> root         209       2  0 May07 ?        00:00:00 [irq/44-pciehp]
>>>>>> root         210       2  0 May07 ?        00:00:00 [irq/45-pciehp]
>>>>>> root         211       2  0 May07 ?        00:00:00 [irq/46-pciehp]
>>>>>> root         212       2  0 May07 ?        00:00:00 [irq/47-pciehp]
>>>>>> root         213       2  0 May07 ?        00:00:00 [irq/48-pciehp]
>>>>>> root         214       2  0 May07 ?        00:00:00 [irq/49-pciehp]
>>>>>> root         215       2  0 May07 ?        00:00:00 [irq/50-pciehp]
>>>>>> root         216       2  0 May07 ?        00:00:00 [irq/51-pciehp]
>>>>>> root         217       2  0 May07 ?        00:00:00 [irq/52-pciehp]
>>>>>> root         218       2  0 May07 ?        00:00:00 [irq/53-pciehp]
>>>>>> root         219       2  0 May07 ?        00:00:00 [irq/54-pciehp]
>>>>>> root         220       2  0 May07 ?        00:00:00 [irq/55-pciehp]
>>>>>> root         222       2  0 May07 ?        00:00:00 [acpi_thermal_pm]
>>>>>> root         223       2  0 May07 ?        00:00:00 [kmpath_rdacd]
>>>>>> root         224       2  0 May07 ?        00:00:00 [kaluad]
>>>>>> root         226       2  0 May07 ?        00:00:00 [ipv6_addrconf]
>>>>>> root         227       2  0 May07 ?        00:00:00 [kstrp]
>>>>>> root         508       2  0 May07 ?        00:00:00 [mpt_poll_0]
>>>>>> root         509       2  0 May07 ?        00:00:00 [ata_sff]
>>>>>> root         510       2  0 May07 ?        00:00:00 [mpt/0]
>>>>>> root         512       2  0 May07 ?        00:00:00 [scsi_eh_0]
>>>>>> root         516       2  0 May07 ?        00:00:00 [scsi_tmf_0]
>>>>>> root         517       2  0 May07 ?        00:00:00 [scsi_eh_1]
>>>>>> root         518       2  0 May07 ?        00:00:00 [scsi_tmf_1]
>>>>>> root         524       2  0 May07 ?        00:00:14 [irq/16-vmwgfx]
>>>>>> root         525       2  0 May07 ?        00:00:00 [ttm_swap]
>>>>>> root         531       2  0 May07 ?        00:00:00 [scsi_eh_2]
>>>>>> root         532       2  0 May07 ?        00:00:00 [scsi_tmf_2]
>>>>>> root         540       2  0 May07 ?        00:00:01
>>>> [kworker/7:1H-kblockd]
>>>>>> root         541       2  0 May07 ?        00:00:01
>>>> [kworker/5:1H-kblockd]
>>>>>> root         567       2  0 May07 ?        00:00:07 [jbd2/sda3-8]
>>>>>> root         568       2  0 May07 ?        00:00:00 [ext4-rsv-conver]
>>>>>> root         611       2  0 May07 ?        00:00:49 [jbd2/sda2-8]
>>>>>> root         612       2  0 May07 ?        00:00:00 [ext4-rsv-conver]
>>>>>> root         613       2  0 May07 ?        00:00:01
>>>> [kworker/6:1H-kblockd]
>>>>>> root         650       2  0 May07 ?        00:00:01
>>>> [kworker/4:1H-kblockd]
>>>>>> root         659       2  0 May07 ?        00:00:02
>>>> [kworker/3:1H-kblockd]
>>>>>> root         660       2  0 May07 ?        00:00:02
>>>> [kworker/2:1H-kblockd]
>>>>>> root         662       2  0 May07 ?        00:00:01
>>>> [kworker/0:1H-kblockd]
>>>>>> root         679       1  0 May07 ?        00:00:03
>>>> /usr/lib/systemd/systemd-journald
>>>>>> root         682       2  0 May07 ?        00:00:02
>>>> [kworker/1:1H-kblockd]
>>>>>> root         716       1  0 May07 ?        00:00:01
>>>> /usr/lib/systemd/systemd-udevd
>>>>>> root         723       2  0 May07 ?        00:00:18 [jbd2/sdb1-8]
>>>>>> root         724       2  0 May07 ?        00:00:00 [ext4-rsv-conver]
>>>>>> root         733       2  0 May07 ?        00:00:14 [jbd2/sda6-8]
>>>>>> root         734       2  0 May07 ?        00:00:00 [ext4-rsv-conver]
>>>>>> root         738       2  0 May07 ?        00:00:00 [jbd2/sdc1-8]
>>>>>> root         739       2  0 May07 ?        00:00:00 [ext4-rsv-conver]
>>>>>> root         744       2  0 May07 ?        00:00:00 [jbd2/sda1-8]
>>>>>> root         745       2  0 May07 ?        00:00:00 [ext4-rsv-conver]
>>>>>> root         776       1  0 May07 ?        00:00:01 /sbin/auditd
>>>>>> root         790       2  0 May07 ?        00:00:00 [rpciod]
>>>>>> root         791       2  0 May07 ?        00:00:00 [kworker/u17:0]
>>>>>> root         792       2  0 May07 ?        00:00:00 [xprtiod]
>>>>>> dbus         883       1  0 May07 ?        00:00:03
>>>> /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile 
>>>> --systemd-activation --syslog-only
>>>>>> root         885       1  0 May07 ?        00:02:42
>>>> /usr/sbin/NetworkManager --no-daemon
>>>>>> root         887       1  0 May07 ?        00:00:44
>>>> /usr/sbin/irqbalance --foreground
>>>>>> root         888       1  0 May07 ?        00:00:00
>>>> /usr/bin/VGAuthService -s
>>>>>> root         889       1  0 May07 ?        00:10:28 /usr/bin/vmtoolsd
>>>>>> root         890       1  0 May07 ?        00:00:00 /usr/sbin/sssd -i
>>>> --logger=files
>>>>>> chrony       906       1  0 May07 ?        00:00:00 /usr/sbin/chronyd
>>>>>> rngd         944       1  0 May07 ?        00:00:49 /sbin/rngd -f
>>>> --fill-watermark=0
>>>>>> root         946     890  0 May07 ?        00:00:06
>>>> /usr/libexec/sssd/sssd_be --domain implicit_files --uid 0 --gid 0 
>>>> --logger=files
>>>>>> root         954       1  0 May07 ?        00:00:00 /usr/sbin/iscsid -f
>>>> -d2
>>>>>> root         962     890  0 May07 ?        00:00:12
>>>> /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --logger=files
>>>>>> root         963       1  0 May07 ?        00:00:00 /usr/sbin/gssproxy
>>>> -D
>>>>>> root         970       1  0 May07 ?        00:00:02
>>>> /usr/lib/systemd/systemd-logind
>>>>>> root         971       2  0 May07 ?        00:00:00 [iscsi_eh]
>>>>>> root         998       1  0 May07 ?        00:00:33 /usr/sbin/rsyslogd
>>>> -n
>>>>>> root        1007       1  0 May07 ?        00:00:00 /usr/sbin/crond -n
>>>>>> root        6223       1  0 May07 ?        00:07:27
>>>> /usr/local/vtl/bin/sanidmgr
>>>>>> root        6271       2  0 May07 ?        00:00:03 [bksc_exec]
>>>>>> root        6272       2  0 May07 ?        00:00:03 [bksc_done]
>>>>>> root        6274       2  0 May07 ?        00:00:00 [scsi_eh_3]
>>>>>> root        6275       2  0 May07 ?        00:00:00 [scsi_tmf_3]
>>>>>> root        6460       2  0 May07 ?        00:00:00 [kdownstream]
>>>>>> root        6461       2  0 May07 ?        00:00:07 [msgfilter]
>>>>>> root        6463       2  0 May07 ?        00:00:12 [iocore]
>>>>>> root        6466       2  0 May07 ?        00:00:02 [ioctl_mgr]
>>>>>> root        6467       2  0 May07 ?        00:00:02 [ioctl_mgr]
>>>>>> root        6505       1  0 May07 ?        00:00:00 fsnupd -d 0
>>>>>> root        6523       2  0 May07 ?        00:01:50 [trudpd]
>>>>>> root        6539       2  0 May07 ?        00:00:00 [fsnalias_worker]
>>>>>> root        6541       2  0 May07 ?        00:00:00 [fsnalias_check]
>>>>>> root        6551       2  0 May07 ?        00:00:00 [misc_thread]
>>>>>> root        6553       2  0 May07 ?        00:00:01 [mtcp2_listen]
>>>>>> root        6554       2  0 May07 ?        00:00:00 [listen_handle]
>>>>>> root        6557       2  0 May07 ?        00:00:00 [ipstord]
>>>>>> root        6579       2  0 May07 ?        00:00:00 [kfsvshostwq]
>>>>>> root        6580       2  0 May07 ?        00:00:00 [scsi_eh_4]
>>>>>> root        6582       2  0 May07 ?        00:00:00 [scsi_tmf_4]
>>>>>> root        6654       1  0 May07 ?        00:00:00
>>>> /usr/local/vtl/bin/ipstorcomm -d 0
>>>>>> root        6655    6654  0 May07 ?        00:05:18
>>>> /usr/local/vtl/bin/ipstorcomm -d 0
>>>>>> root        6656    6655  0 May07 ?        00:00:36
>>>> /usr/local/vtl/bin/ipstorcomm -d 0
>>>>>> root        6693       1  0 May07 ?        00:00:00 rdesirkey
>>>>>> root        6699       1  0 May07 ?        00:00:12 ikmd
>>>>>> root        6721       1  0 May07 ?        00:00:35
>>>> /usr/local/vtl/bin/iscliproxy
>>>>>> root        6747       1  0 May07 ?        00:00:40
>>>> /usr/local/vtl/bin/ipstorlog
>>>>>> root        6780       1  0 May07 ?        00:04:50
>>>> /usr/local/vtl/bin/snmpd
>>>>>> root        7722       1  0 May07 ?        00:00:00
>>>> /usr/local/vtl/bin/ipstorsm
>>>>>> root        7723    7722  0 May07 ?        01:08:12
>>>> /usr/local/vtl/bin/ipstorsm
>>>>>> root        7901       1  0 May07 ?        00:00:00
>>>> /usr/lib/systemd/systemd --user
>>>>>> root        7904    7901  0 May07 ?        00:00:00 (sd-pam)
>>>>>> root        8179       1  0 May07 tty1     00:00:00 /sbin/agetty -o -p
>>>> -- \u --noclear tty1 linux
>>>>>> root       14199       1  0 May08 ?        00:00:01 sshd: root [priv]
>>>>>> root       14201   14199  0 May08 ?        00:00:01 sshd: root@pts/0
>>>>>> root       14202   14201  0 May08 pts/0    00:00:00 -bash
>>>>>> root       14261       1  0 May08 ?        00:00:00 /usr/sbin/sshd -D
>>>> -oCiphers=aes256-...@openssh.com,chacha20-poly1...@openssh.com
>>>> ,aes256-ctr,aes256-cbc,aes128-...@openssh.com,aes128-ctr,aes128-cbc
>>>> -oMACs=hmac-sha2-256-...@openssh.com,hmac-sha1-...@openssh.com,
>>>> umac-128-...@openssh.com,hmac-sha2-512-...@openssh.com
>>>> ,hmac-sha2-256,hmac-sha1,umac-...@openssh.com,hmac-sha2-512
>>>> -oGSSAPIKexAlgorithms=gss-curve25519-sha256-,gss-nistp256-sha256-,gss
>>>> -
>>>> group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-
>>>> -oKexAlgorithms=curve25519-sha256,curve25519-sha...@libssh.org,ecdh-s
>>>> h 
>>>> a2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-grou
>>>> p
>>>> -exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16
>>>> - 
>>>> sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sh
>>>> a
>>>> 1,diffie-hellman-group14-sha1 
>>>> -oHostKeyAlgorithms=ecdsa-sha2-nistp256,
>>>> ecdsa-sha2-nistp256-cert-...@openssh.com,ecdsa-sha2-nistp384,
>>>> ecdsa-sha2-nistp384-cert-...@openssh.com,ecdsa-sha2-nistp521,
>>>> ecdsa-sha2-nistp521-cert-...@openssh.com,ssh-ed25519,
>>>> ssh-ed25519-cert-...@openssh.com,rsa-sha2-256,
>>>> rsa-sha2-256-cert-...@openssh.com,rsa-sha2-512,
>>>> rsa-sha2-512-cert-...@openssh.com,ssh-rsa,ssh-rsa-cert-...@openssh.co
>>>> m -oPubkeyAcceptedKeyTypes=ecdsa-sha2-nistp256,
>>>> ecdsa-sha2-nistp256-cert-...@openssh.com,ecdsa-sha2-nistp384,
>>>> ecdsa-sha2-nistp384-cert-...@openssh.com,ecdsa-sha2-nistp521,
>>>> ecdsa-sha2-nistp521-cert-...@openssh.com,ssh-ed25519,
>>>> ssh-ed25519-cert-...@openssh.com,rsa-sha2-256,
>>>> rsa-sha2-256-cert-...@openssh.com,rsa-sha2-512,
>>>> rsa-sha2-512-cert-...@openssh.com,ssh-rsa,ssh-rsa-cert-...@openssh.co
>>>> m
>>>> -oCASignatureAlgorithms=ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa
>>>> - sha2-nistp521,ssh-ed25519,rsa-sha2-256,rsa-sha2-512,ssh-rsa
>>>>>> root      122905       2  0 May19 ?        00:00:27 [kworker/7:0-events]
>>>>>> root      126037       2  0 05:30 ?        00:00:00 [kworker/1:2-events]
>>>>>> root      128108       2  0 08:40 ?        00:00:00
>>>> [kworker/4:1-events_power_efficient]
>>>>>> root      128360       2  0 09:10 ?        00:00:00
>>>> [kworker/4:2-cgroup_destroy]
>>>>>> root      128430       2  0 09:30 ?        00:00:00
>>>> [kworker/6:0-mm_percpu_wq]
>>>>>> root      128738       2  0 11:00 ?        00:00:00
>>>> [kworker/3:3-mm_percpu_wq]
>>>>>> root      128761       2  0 11:00 ?        00:00:00
>>>> [kworker/2:1-mm_percpu_wq]
>>>>>> root      130244       1  1 11:01 ?        00:03:50
>>>> /usr/local/jre/bin/java
>>>> -Djava.util.logging.config.file=/usr/local/apache-tomcat/conf/logging.
>>>> properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 
>>>> -d64 -server -Xms1800m -Xmx8192m -XX:MaxMetaspaceSize=1800m
>>>> -Djdk.tls.ephemeralDHKeySize=2048
>>>> -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
>>>> -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
>>>> -Dignore.endorsed.dirs= -classpath
>>>> /usr/local/apache-tomcat/bin/bootstrap.jar:/usr/local/apache-tomcat/b
>>>> i n/tomcat-juli.jar -Dcatalina.base=/usr/local/apache-tomcat
>>>> -Dcatalina.home=/usr/local/apache-tomcat
>>>> -Djava.io.tmpdir=/usr/local/apache-tomcat/temp
>>>> org.apache.catalina.startup.Bootstrap start
>>>>>> elastic+  130516       1  1 11:01 ?        00:03:26 /bin/java -Xms8g
>>>> -Xmx8g -Djava.awt.headless=true -XX:+UseParNewGC 
>>>> -XX:+UseConcMarkSweepGC
>>>> -XX:CMSInitiatingOccupancyFraction=75
>>>> -XX:+UseCMSInitiatingOccupancyOnly
>>>> -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC
>>>> -Dfile.encoding=UTF-8 -Delasticsearch -Des.foreground=yes 
>>>> -Des.path.home=/usr/share/elasticsearch -cp 
>>>> :/usr/share/elasticsearch/lib/elasticsearch-1.7.6.jar:/usr/share/elas
>>>> t
>>>> icsearch/lib/*:/usr/share/elasticsearch/lib/sigar/*
>>>> -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid
>>>> -Des.default.path.home=/usr/share/elasticsearch
>>>> -Des.default.path.logs=/var/log/elasticsearch
>>>> -Des.default.path.data=/var/lib/elasticsearch
>>>> -Des.default.config=/etc/elasticsearch/elasticsearch.yml
>>>> -Des.default.path.conf=/etc/elasticsearch
>>>> org.elasticsearch.bootstrap.Elasticsearch
>>>>>> postgres  130548       1  0 11:01 ?        00:00:00
>>>> /usr/pgsql-9.6/bin/postgres -D /DB/postgres/data
>>>>>> postgres  130561  130548  0 11:01 ?        00:00:00 postgres: logger
>>>> process
>>>>>> postgres  130563  130548  0 11:01 ?        00:00:00 postgres:
>>>> checkpointer process
>>>>>> postgres  130564  130548  0 11:01 ?        00:00:00 postgres: writer
>>>> process
>>>>>> postgres  130565  130548  0 11:01 ?        00:00:00 postgres: wal
>>>> writer process
>>>>>> postgres  130566  130548  0 11:01 ?        00:00:00 postgres:
>>>> autovacuum launcher process
>>>>>> postgres  130567  130548  0 11:01 ?        00:00:00 postgres: stats
>>>> collector process
>>>>>> root      130570       1  0 11:01 ?        00:00:01
>>>> /usr/local/freestor/bin/fmsd
>>>>>> postgres  130675  130548  0 11:01 ?        00:00:03 postgres: postgres
>>>> freestor 127.0.0.1(42660) idle
>>>>>> postgres  130676  130548  0 11:01 ?        00:00:04 postgres: postgres
>>>> freestor 127.0.0.1(42662) idle
>>>>>> postgres  130677  130548  0 11:01 ?        00:00:03 postgres: postgres
>>>> freestor 127.0.0.1(42664) idle
>>>>>> root      131206       2  0 13:00 ?        00:00:00 [kworker/7:2]
>>>>>> root      131269       2  0 13:20 ?        00:00:00 [kworker/6:2]
>>>>>> root      131347       2  0 13:49 ?        00:00:00
>>>> [kworker/5:1-mm_percpu_wq]
>>>>>> root      131364       2  0 13:56 ?        00:00:00
>>>> [kworker/u16:2-events_unbound]
>>>>>> root      131401       2  0 14:05 ?        00:00:00
>>>> [kworker/u16:1-events_unbound]
>>>>>> root      131438       2  0 14:20 ?        00:00:00 [kworker/2:2]
>>>>>> root      131465       2  0 14:30 ?        00:00:00 [kworker/0:2-events]
>>>>>> root      131490       2  0 14:40 ?        00:00:00 [kworker/3:0]
>>>>>> root      131491       2  0 14:40 ?        00:00:00 [kworker/0:1-events]
>>>>>> root      131515       2  0 14:49 ?        00:00:00
>>>> [kworker/1:0-events_freezable]
>>>>>> root      131520       2  0 14:50 ?        00:00:00 [kworker/5:0]
>>>>>> root      131525   14261  0 14:52 ?        00:00:00 sshd: root [priv]
>>>>>> root      131529  131525  0 14:52 ?        00:00:00 sshd: root@pts/1
>>>>>> root      131530  131529  0 14:52 pts/1    00:00:00 -bash
>>>>>> root      131600  131530  0 14:57 pts/1    00:00:00 ps -ef
>>>>>>
>>>>>> Thanks
>>>>>> -----Original Message-----
>>>>>> From: Christopher Schultz <ch...@christopherschultz.net>
>>>>>> Sent: Thursday, May 20, 2021 12:49 PM
>>>>>> To: users@tomcat.apache.org
>>>>>> Subject: Re: apache-tomcat-8.5.59 too many open files on Linux 8
>>>>>>
>>>>>> Yeggy,
>>>>>>
>>>>>> On 5/19/21 19:37, Yeggy Javadi wrote:
>>>>>>> Hi Chris,
>>>>>>> Thanks for your email. Below are replies to your questions:
>>>>>>>
>>>>>>> 1. Did you upgrade anything recently (like Java VM)?
>>>>>>> [YJ] To support Linux 8, only Postgres was upgraded from version
>>>>>>> 9.3
>>>> to 9.6.
>>>>>>> 2. What is error you are seeing? A full stack trace would be helpful.
>>>>>>> [YJ] The application error can occur in any place with the (Too 
>>>>>>> many
>>>> open files) error when the limit of open files for tomcat has reached
>>>> (262144 in my environment), for example, I can get a connectivity 
>>>> error when pulling info from a server as below :
>>>>>>>      [InventoryPullerTask - 10.1.6.25] ERROR 
>>>>>>> FSSDataCollectorService - Error : The XML configuration file 
>>>>>>> failed to be retrieved for server 10.1.6.25. Check server 
>>>>>>> connectivity.Error retrieving IPStorConfg for
>>>>>>> server=10.1.6.25 error=java.io.FileNotFoundException:
>>>>>>> /usr/local/apache-tomcat-8.5.59/webapps/ROOT/WEB-INF/lib/spring-orm-3.
>>>>>>> 2.10.RELEASE.jar (Too many open files) restatus=-1 output=
>>>>>>>
>>>>>>> 3. What is your <Connector> configuration?
>>>>>>> [YJ] It is as below in server.xml:
>>>>>>>
>>>>>>>         <Connector address="0.0.0.0" port="80" protocol="HTTP/1.1"
>>>>>>>                    maxThreads="150"
>>>>>>>                    minSpareThreads="25" maxSpareThreads="75"
>>>>>>>                    enableLookups="false" acceptCount="100"
>>>>>>>                    connectionTimeout="20000" disableUploadTimeout="true"
>>>>>>>                    compression="on"
>>>>>>>                    compressionMinSize="2048"
>>>>>>>                    noCompressionUserAgents="gozilla, traviata"
>>>>>>>
>>>>   compressableMimeType="text/html,text/xml,application/json"
>>>>>>>                    redirectPort="443" /> ...........
>>>>>>>         <Connector address="0.0.0.0" port="443"
>>>> protocol="org.apache.coyote.http11.Http11NioProtocol" SSLEnabled="true"
>>>>>>>                    maxThreads="150" scheme="https" secure="true"
>>>>>>>                    clientAuth="false" sslEnabledProtocols="TLSv1.2"
>>>>>>>
>>>>   
>>>> ciphers="TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA"
>>>>>>>                    keystoreFile="conf/freestor"
>>>> keystorePass="freestor"/>
>>>>>>>         <!-- Define an AJP 1.3 Connector on port 8009 -->
>>>>>>>         <Connector address="0.0.0.0" port="8009" protocol="AJP/1.3"
>>>>>>> redirectPort="443" secretRequired="false" />
>>>>>> You have lots of settings on the HTTP connector but not the HTTPS one.
>>>>>> Is that intentional? Which one are you using? Or both?
>>>>>>
>>>>>> Are you using AJP? If not, disable the connetor entirely.
>>>>>> Definitely
>>>> never enable it on 0.0.0.0 unless you really know what you are doing.
>>>>>>> 4. Are you counting all the open files for a single process or all 
>>>>>>> of
>>>> the sub-processes which represent the threads of the main process?
>>>>>>> Different kernel versions count things differently.
>>>>>>> [YJ] I am just getting the process ID of tomcat and count open 
>>>>>>> files for that process
>>>>>> Ok.
>>>>>>
>>>>>>> 5. Running lsof, netstat, etc. can you see if some large number of
>>>> those sockets are bound to any specific port (listen or connect)?
>>>>>>> [YJ] Here is the netstat output:
>>>>>>> Active Internet connections (w/o servers)
>>>>>>> Proto Recv-Q Send-Q Local Address           Foreign Address
>>>>   State
>>>>>>> tcp        0      0 Yeggy-F8-FMSVA:ssh      10.12.3.33:55236
>>>> ESTABLISHED
>>>>>>> tcp        0     64 Yeggy-F8-FMSVA:ssh      10.197.255.10:60378
>>>>   ESTABLISHED
>>>>>>> tcp        0      0 localhost.loca:postgres localhost.localdo:36846
>>>> ESTABLISHED
>>>>>> That's not helpful. I want to see what the JVM process is holding, 
>>>>>> not
>>>> what the whole OS is handling.
>>>>>> My guess is that lsof is going to be a better bet for inspecting 
>>>>>> the
>>>> process. But if you get "too many open files" being thrown inside the 
>>>> JVM and you don't see a 1/4 million lines of "lsof" output on your 
>>>> process, then something else is going on. But remember to limit your 
>>>> lsof to *just the JVM* process.
>>>>>> -chris
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Christopher Schultz <ch...@christopherschultz.net>
>>>>>>> Sent: Monday, May 17, 2021 5:18 PM
>>>>>>> To: users@tomcat.apache.org
>>>>>>> Subject: Re: apache-tomcat-8.5.59 too many open files on Linux 8
>>>>>>>
>>>>>>> Yeggy,
>>>>>>>
>>>>>>> On 5/17/21 15:17, Yeggy Javadi wrote:
>>>>>>>> Hi,
>>>>>>>> I am seeing too many TCP sockets open by tomcat on Oracle Linux 8:
>>>>>>>> java    51798 root 6601u     sock                0,9      0t0 3959342
>>>> protocol: TCPv6
>>>>>>>> I was running the same application on Linux 7.6 with the same 
>>>>>>>> apache-tomcat version without seeing this issue. I saw that 
>>>>>>>> RedHat has removed apache-tomcat fom their Linux 8 distribution.
>>>>>>>> Please help with throubleshooting and indicating if there is any 
>>>>>>>> changes in Linux
>>>>>>>> 8 that can explain this
>>>>>>>>
>>>>>>>> Configuration:
>>>>>>>> apache-tomcat-8.5.59
>>>>>>>> OEL Linux 8 kernel 4.18.0-193.28.1.el8_2.x86_64 Web-based 
>>>>>>>> application using Postgres and elastic search databases
>>>>>>>>
>>>>>>>>
>>>>>>>> # ps -ef | grep tomcat
>>>>>>>> root       51798       1  1 11:16 ?        00:06:01
>>>> /usr/local/jre/bin/java
>>>> -Djava.util.logging.config.file=/usr/local/apache-tomcat/conf/logging.
>>>> properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 
>>>> -d64 -server -Xms1800m -Xmx8192m -XX:MaxMetaspaceSize=1800m
>>>> -Djdk.tls.ephemeralDHKeySize=2048
>>>> -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
>>>> -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
>>>> -Dignore.endorsed.dirs= -classpath
>>>> /usr/local/apache-tomcat/bin/bootstrap.jar:/usr/local/apache-tomcat/b
>>>> i n/tomcat-juli.jar -Dcatalina.base=/usr/local/apache-tomcat
>>>> -Dcatalina.home=/usr/local/apache-tomcat
>>>> -Djava.io.tmpdir=/usr/local/apache-tomcat/temp
>>>> org.apache.catalina.startup.Bootstrap start
>>>>>>>> root       54122   54037  0 19:43 pts/1    00:00:00 grep --color=auto
>>>> tomcat
>>>>>>>> # cat /proc/51798/limits
>>>>>>>> Limit                     Soft Limit           Hard Limit
>>>>   Units
>>>>>>>> Max cpu time              unlimited            unlimited
>>>> seconds
>>>>>>>> Max file size             unlimited            unlimited
>>>> bytes
>>>>>>>> Max data size             unlimited            unlimited
>>>> bytes
>>>>>>>> Max stack size            8388608              unlimited
>>>> bytes
>>>>>>>> Max core file size        unlimited            unlimited
>>>> bytes
>>>>>>>> Max resident set          unlimited            unlimited
>>>> bytes
>>>>>>>> Max processes             256518               256518
>>>>   processes
>>>>>>>> Max open files            262144               262144
>>>>   files
>>>>>>>> Max locked memory         65536                65536
>>>> bytes
>>>>>>>> Max address space         unlimited            unlimited
>>>> bytes
>>>>>>>> Max file locks            unlimited            unlimited
>>>> locks
>>>>>>>> Max pending signals       256518               256518
>>>>   signals
>>>>>>>> Max msgqueue size         819200               819200
>>>>   bytes
>>>>>>>> Max nice priority         0                    0
>>>>>>>> Max realtime priority     0                    0
>>>>>>>> Max realtime timeout      unlimited            unlimited            us
>>>>>>>>
>>>>>>>>
>>>>>>>> Too may open files:
>>>>>>>> # lsof -p 51798 | wc -l
>>>>>>>> 6679
>>>>>>>>
>>>>>>>>
>>>>>>>> Most of open files are TCP sockets:
>>>>>>>> java    51798 root 6601u     sock                0,9      0t0 3959342
>>>> protocol: TCPv6
>>>>>>>> # lsof -p 51798 | grep "protocol: TCPv6" | wc -l
>>>>>>>> 6413
>>>>>>> Some potentially dumb questions:
>>>>>>>
>>>>>>> 1. Did you upgrade anything recently (like Java VM)?
>>>>>>>
>>>>>>> 2. What is error you are seeing? A full stack trace would be helpful.
>>>>>>>
>>>>>>> 3. What is your <Connector> configuration?
>>>>>>>
>>>>>>> 4. Are you counting all the open files for a single process or all 
>>>>>>> of
>>>> the sub-processes which represent the threads of the main process?
>>>>>>> Different kernel versions count things differently.
>>>>>>>
>>>>>>> 5. Running lsof, netstat, etc. can you see if some large number of
>>>> those sockets are bound to any specific port (listen or connect)?
>>>>>>> -chris
>>>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>>
>>> --
>>> Noelette Stout
>>>
>>> ITS Enterprise Applications - Application Administrator - Senior
>>>
>>> Business Administration Building, Rm 109L
>>> 921 South 8th Ave 8037
>>> Idaho State University
>>> Pocatello ID 83209
>>> E-mail: stounoel "at" isu "dot" edu
>>> Desk: 208-282-2554
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>

Attachment: OpenPGP_signature
Description: OpenPGP digital signature

Reply via email to