Hi,
I am using Knox server gateway (version 0.12.0) with Hortonworks
HDP-2.6.2.0 . There seems to be a memory leak with the knox process
because of which it keeps acquiring more and more memory. My test workload
does very frequent webhdfs rest apis from hundreds of connections. When the
workload is executing, I notice that the knox gateway process memory keeps
increasing.
PID VSZ SZ RSS %CPU TIME CMD
24703 44988224 11247056 2195740 78.8 01:37:02 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44988224 11247056 2195740 78.8 01:37:03 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44989420 11247355 2335764 84.3 01:44:13 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44989576 11247394 2326344 89.5 01:51:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44989744 11247436 2345940 94.8 01:58:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/ga
teway.jar
24703 44989908 11247477 2515100 100 02:05:13 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990072 11247518 2653252 105 02:12:18 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990244 11247561 2777268 110 02:19:26 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878924 113 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878924 112 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878872 112 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878872 111 02:23:04 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878792 111 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878792 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878748 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878748 110 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878660 109 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878660 109 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878580 108 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878580 108 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878500 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878500 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878392 107 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878392 106 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878348 106 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878348 105 02:23:05 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878268 105 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878268 105 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878124 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878124 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878016 104 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
24703 44990324 11247581 2878016 103 02:23:06 /usr/jdk64/jdk1.8.0_112/bin/java
-Djava.library.path=/usr/hdp/current/knox-server/ext/native -jar
/usr/hdp/current/knox-server/bin/gat
eway.jar
Has anyone seen this behaviour with knox ?
Following is my Knox topology, I am using LDAP authentication
<topology>
<gateway>
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>sessionTimeout</name>
<value>30</value>
</param>
<param>
<name>main.ldapRealm</name>
<value>org.apache.hadoop.gateway.shirorealm.
KnoxLdapRealm</value>
</param>
<param>
<name>main.ldapRealm.userDnTemplate</name>
<value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</
value>
</param>
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://{{knox_host_name}}:33389</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.
authenticationMechanism</name>
<value>simple</value>
</param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
</provider>
<provider>
<role>authorization</role>
<name>AclsAuthz</name>
<enabled>true</enabled>
<param>
<name>knox.acl</name>
<value>admin;*;*</value>
</param>
</provider>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
</gateway>
<service>
<role>KNOX</role>
</service>
</topology>
Thanks,
Vaibhav Kulkarni
