[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377657#comment-14377657
]
Hudson commented on YARN-3336:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #876 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/876/])
YARN-3336. FileSystem memory leak in DelegationTokenRenewer. (cnauroth: rev
6ca1f12024fd7cec7b01df0f039ca59f3f365dc1)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377932#comment-14377932
]
Hudson commented on YARN-3336:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2074 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2074/])
YARN-3336. FileSystem memory leak in DelegationTokenRenewer. (cnauroth: rev
6ca1f12024fd7cec7b01df0f039ca59f3f365dc1)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377875#comment-14377875
]
Hudson commented on YARN-3336:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2092 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2092/])
YARN-3336. FileSystem memory leak in DelegationTokenRenewer. (cnauroth: rev
6ca1f12024fd7cec7b01df0f039ca59f3f365dc1)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377641#comment-14377641
]
Hudson commented on YARN-3336:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #142 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/142/])
YARN-3336. FileSystem memory leak in DelegationTokenRenewer. (cnauroth: rev
6ca1f12024fd7cec7b01df0f039ca59f3f365dc1)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377952#comment-14377952
]
Hudson commented on YARN-3336:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #133 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/133/])
YARN-3336. FileSystem memory leak in DelegationTokenRenewer. (cnauroth: rev
6ca1f12024fd7cec7b01df0f039ca59f3f365dc1)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378006#comment-14378006
]
Hudson commented on YARN-3336:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #142 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/142/])
YARN-3336. FileSystem memory leak in DelegationTokenRenewer. (cnauroth: rev
6ca1f12024fd7cec7b01df0f039ca59f3f365dc1)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376447#comment-14376447
]
zhihai xu commented on YARN-3336:
-
Thanks [~cnauroth] for valuable feedback and committing the patch! Greatly
appreciated.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376300#comment-14376300
]
Hudson commented on YARN-3336:
--
FAILURE: Integrated in Hadoop-trunk-Commit #7405 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7405/])
YARN-3336. FileSystem memory leak in DelegationTokenRenewer. (cnauroth: rev
6ca1f12024fd7cec7b01df0f039ca59f3f365dc1)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376914#comment-14376914
]
Chris Nauroth commented on YARN-3336:
-
[~zxu], I apologize, but I missed entering your name on the git commit message:
{code}
commit 6ca1f12024fd7cec7b01df0f039ca59f3f365dc1
Author: cnauroth cnaur...@apache.org
Date: Mon Mar 23 10:45:50 2015 -0700
YARN-3336. FileSystem memory leak in DelegationTokenRenewer.
{code}
Unfortunately, this isn't something we can change, because it could mess up the
git history.
You're still there in CHANGES.txt though, so you get the proper credit for the
patch:
{code}
YARN-3336. FileSystem memory leak in DelegationTokenRenewer.
(Zhihai Xu via cnauroth)
{code}
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377258#comment-14377258
]
zhihai xu commented on YARN-3336:
-
[~cnauroth], Not a problem, thanks for the notification.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Fix For: 2.7.0
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372434#comment-14372434
]
Hadoop QA commented on YARN-3336:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706081/YARN-3336.004.patch
against trunk revision fe5c23b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7051//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7051//console
This message is automatically generated.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372290#comment-14372290
]
zhihai xu commented on YARN-3336:
-
Oh, that is a good idea. I uploaded a new patch YARN-3336.004.patch, which use
the instance counter for the verification. Please review it.
many thanks [~cnauroth]!
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch, YARN-3336.004.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372127#comment-14372127
]
Chris Nauroth commented on YARN-3336:
-
Hello, [~zxu]. Thank you for providing the new patch and adding the test.
I think we can avoid the changes in {{FileSystem}} by adding an instance
counter to {{MyFS}}. We can increment it in the constructor and decrement it
in {{close}}. Then, the test can get the value of the counter before making
the calls to {{obtainSystemTokensForUser}} and assert that the counter has the
same value after those calls.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360805#comment-14360805
]
zhihai xu commented on YARN-3336:
-
TestRMWebServices and TestFairSchedulerQueueACLs passed in my local latest
build and both test failures are not related to my patch.
{code}
---
T E S T S
---
Running
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 53.871 sec - in
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
---
T E S T S
---
Running org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.004 sec - in
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
Results :
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0
{code}
The findbugs warnings are also not related to my patch. YARN-3341 is to fix one
of the findbugs warnings.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360025#comment-14360025
]
zhihai xu commented on YARN-3336:
-
I uploaded a new patch YARN-3336.003.patch to fix the test failure due to the
change in FileSystem.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360141#comment-14360141
]
Hadoop QA commented on YARN-3336:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704376/YARN-3336.003.patch
against trunk revision 387f271.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6951//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6951//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6951//console
This message is automatically generated.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch, YARN-3336.003.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358291#comment-14358291
]
zhihai xu commented on YARN-3336:
-
Hi [~cnauroth], Yes, you are right, I forget the FileSystem RPC Proxy also need
correct ugi information.
I uploaded a new patch YARN-3336.001.patch which addressed your comment. Please
review it.
many thanks for the review.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358390#comment-14358390
]
Hadoop QA commented on YARN-3336:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704121/YARN-3336.001.patch
against trunk revision ff83ae7.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestRM
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6935//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6935//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6935//console
This message is automatically generated.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359224#comment-14359224
]
Hadoop QA commented on YARN-3336:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704208/YARN-3336.001.patch
against trunk revision 06ce1d9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
org.apache.hadoop.yarn.server.resourcemanager.TestClientRMTokens
org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
The following test timeouts occurred in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestRMRestTestTests
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorizatTestTests
org.apache.hadoop.yarn.server.resourcemanager.TestFifoScheTests
org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMReTests
org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCRespoTests
org.apache.hadoop.yarn.server.resourcemanager.TestContainerResourcTests
org.apache.hadoop.yarn.server.resourcemanager.TestResourceManTests
org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6941//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6941//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6941//console
This message is automatically generated.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359181#comment-14359181
]
zhihai xu commented on YARN-3336:
-
The findbugs warning is not related to my change.
I just created YARN-3341 to fix this findbugs warning.
The TestRM failure is also not related to my change, which is a timeout failure.
It passed at my local latest build with the following message.
{code}
---
T E S T S
---
Running org.apache.hadoop.yarn.server.resourcemanager.TestRM
Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 166.002 sec -
in org.apache.hadoop.yarn.server.resourcemanager.TestRM
Results :
Tests run: 22, Failures: 0, Errors: 0, Skipped: 0
{code}
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359656#comment-14359656
]
zhihai xu commented on YARN-3336:
-
Hi [~cnauroth], I implemented a unit test in the new patch YARN-3336.002.patch.
Without the fix, you can see the test failure with the following message:
{code}
java.lang.AssertionError: expected:1 but was:4
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer.testFSLeakInObtainSystemTokensForUser(TestDelegationTokenRenewer.java:1041)
{code}
I called obtainSystemTokensForUser three times, without the fix, the FileSystem
Cache Size will increase to 4 from 1.
The unit test proved this memory leak issue.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359779#comment-14359779
]
Hadoop QA commented on YARN-3336:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704295/YARN-3336.002.patch
against trunk revision 8212877.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.fs.TestFilterFileSystem
org.apache.hadoop.fs.TestHarFileSystem
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6948//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6948//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6948//console
This message is automatically generated.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch, YARN-3336.001.patch,
YARN-3336.002.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358146#comment-14358146
]
Chris Nauroth commented on YARN-3336:
-
Hi [~zxu]. Nice catch.
I think the current version of the patch would change the owner of all obtained
delegation tokens from the application submitter to the user running the
ResourceManager daemon (i.e. the yarn user). Instead, can we simply call
{{close}} on the {{FileSystem}} after {{addDelegationTokens}}? Closing a
{{FileSystem}} also has the effect of removing it from the cache. Since we
already know that a new instance is getting created every time through this
code path, I don't think closing the instance can impact any other threads.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357900#comment-14357900
]
zhihai xu commented on YARN-3336:
-
I upload a patch for this issue.
The fix is to call FileSystem.get(getConfig()) outside of the proxyUser.doAs.
So FileSystem.get(getConfig()) is at the current RM user context, it will
return a FileSystem from the FileSystem.CACHE instead of creating a new
FileSystem.
Since the patch is straightforward and a very small change, I think we don't
need a test case for this patch.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
isEqual(this.authority, that.authority)
isEqual(this.ugi, that.ugi)
(this.unique == that.unique);
}
return false;
}
{code}
UserGroupInformation.equals will compare subject by reference.
{code}
public boolean equals(Object o) {
if (o == this) {
return true;
} else if (o == null || getClass() != o.getClass()) {
return false;
} else {
return subject == ((UserGroupInformation) o).subject;
}
}
{code}
So in this case, every time createProxyUser and FileSystem.get(getConfig())
are called, a new FileSystem will be created and a new entry will be added to
FileSystem.CACHE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3336) FileSystem memory leak in DelegationTokenRenewer
[
https://issues.apache.org/jira/browse/YARN-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358008#comment-14358008
]
Hadoop QA commented on YARN-3336:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704062/YARN-3336.000.patch
against trunk revision 7a346bc.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6932//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6932//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6932//console
This message is automatically generated.
FileSystem memory leak in DelegationTokenRenewer
Key: YARN-3336
URL: https://issues.apache.org/jira/browse/YARN-3336
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Critical
Attachments: YARN-3336.000.patch
FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new
FileSystem entry will be added to FileSystem#CACHE which will never be
garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
protected Token?[] obtainSystemTokensForUser(String user,
final Credentials credentials) throws IOException, InterruptedException
{
// Get new hdfs tokens on behalf of this user
UserGroupInformation proxyUser =
UserGroupInformation.createProxyUser(user,
UserGroupInformation.getLoginUser());
Token?[] newTokens =
proxyUser.doAs(new PrivilegedExceptionActionToken?[]() {
@Override
public Token?[] run() throws Exception {
return FileSystem.get(getConfig()).addDelegationTokens(
UserGroupInformation.getLoginUser().getUserName(), credentials);
}
});
return newTokens;
}
{code}
The memory leak happened when FileSystem.get(getConfig()) is called with a
new proxy user.
Because createProxyUser will always create a new Subject.
The calling sequence is
FileSystem.get(getConfig())=FileSystem.get(getDefaultUri(conf),
conf)=FileSystem.CACHE.get(uri, conf)=FileSystem.CACHE.getInternal(uri,
conf, key)=FileSystem.CACHE.map.get(key)=createFileSystem(uri, conf)
{code}
public static UserGroupInformation createProxyUser(String user,
UserGroupInformation realUser) {
if (user == null || user.isEmpty()) {
throw new IllegalArgumentException(Null user);
}
if (realUser == null) {
throw new IllegalArgumentException(Null real user);
}
Subject subject = new Subject();
SetPrincipal principals = subject.getPrincipals();
principals.add(new User(user));
principals.add(new RealUser(realUser));
UserGroupInformation result =new UserGroupInformation(subject);
result.setAuthenticationMethod(AuthenticationMethod.PROXY);
return result;
}
{code}
FileSystem#Cache#Key.equals will compare the ugi
{code}
Key(URI uri, Configuration conf, long unique) throws IOException {
scheme = uri.getScheme()==null?:uri.getScheme().toLowerCase();
authority =
uri.getAuthority()==null?:uri.getAuthority().toLowerCase();
this.unique = unique;
this.ugi = UserGroupInformation.getCurrentUser();
}
public boolean equals(Object obj) {
if (obj == this) {
return true;
}
if (obj != null obj instanceof Key) {
Key that = (Key)obj;
return isEqual(this.scheme, that.scheme)
