[jira] [Assigned] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()
[
https://issues.apache.org/jira/browse/HDFS-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erik Krogen reassigned HDFS-12533:
--
Assignee: (was: Erik Krogen)
> NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()
> ---
>
> Key: HDFS-12533
> URL: https://issues.apache.org/jira/browse/HDFS-12533
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Erik Krogen
>Priority: Major
>
> In {{NameNode#getRemoteUser()}}, it first attempts to fetch from the RPC user
> (not a synchronized operation), and if there is no RPC call, it will call
> {{UserGroupInformation#getCurrentUser()}} (which is {{synchronized}}). This
> makes it efficient for RPC operations (the bulk) so that there is not too
> much contention.
> In NNThroughputBenchmark, however, there is no RPC call since we bypass that
> later, so with a high thread count many of the threads are getting stuck. At
> one point I attached a profiler and found that quite a few threads had been
> waiting for {{#getCurrentUser()}} for 2 minutes ( ! ). When taking this away
> I found some improvement in the throughput numbers I was seeing. To more
> closely emulate a real NN we should improve this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()
[
https://issues.apache.org/jira/browse/HDFS-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erik Krogen reassigned HDFS-12533:
--
Assignee: (was: Erik Krogen)
> NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()
> ---
>
> Key: HDFS-12533
> URL: https://issues.apache.org/jira/browse/HDFS-12533
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Erik Krogen
>Priority: Major
>
> In {{NameNode#getRemoteUser()}}, it first attempts to fetch from the RPC user
> (not a synchronized operation), and if there is no RPC call, it will call
> {{UserGroupInformation#getCurrentUser()}} (which is {{synchronized}}). This
> makes it efficient for RPC operations (the bulk) so that there is not too
> much contention.
> In NNThroughputBenchmark, however, there is no RPC call since we bypass that
> later, so with a high thread count many of the threads are getting stuck. At
> one point I attached a profiler and found that quite a few threads had been
> waiting for {{#getCurrentUser()}} for 2 minutes ( ! ). When taking this away
> I found some improvement in the throughput numbers I was seeing. To more
> closely emulate a real NN we should improve this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()
[
https://issues.apache.org/jira/browse/HDFS-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erik Krogen reassigned HDFS-12533:
--
Assignee: Erik Krogen
> NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()
> ---
>
> Key: HDFS-12533
> URL: https://issues.apache.org/jira/browse/HDFS-12533
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
>
> In {{NameNode#getRemoteUser()}}, it first attempts to fetch from the RPC user
> (not a synchronized operation), and if there is no RPC call, it will call
> {{UserGroupInformation#getCurrentUser()}} (which is {{synchronized}}). This
> makes it efficient for RPC operations (the bulk) so that there is not too
> much contention.
> In NNThroughputBenchmark, however, there is no RPC call since we bypass that
> later, so with a high thread count many of the threads are getting stuck. At
> one point I attached a profiler and found that quite a few threads had been
> waiting for {{#getCurrentUser()}} for 2 minutes ( ! ). When taking this away
> I found some improvement in the throughput numbers I was seeing. To more
> closely emulate a real NN we should improve this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()
[
https://issues.apache.org/jira/browse/HDFS-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erik Krogen reassigned HDFS-12533:
--
Assignee: Erik Krogen
> NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()
> ---
>
> Key: HDFS-12533
> URL: https://issues.apache.org/jira/browse/HDFS-12533
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
>
> In {{NameNode#getRemoteUser()}}, it first attempts to fetch from the RPC user
> (not a synchronized operation), and if there is no RPC call, it will call
> {{UserGroupInformation#getCurrentUser()}} (which is {{synchronized}}). This
> makes it efficient for RPC operations (the bulk) so that there is not too
> much contention.
> In NNThroughputBenchmark, however, there is no RPC call since we bypass that
> later, so with a high thread count many of the threads are getting stuck. At
> one point I attached a profiler and found that quite a few threads had been
> waiting for {{#getCurrentUser()}} for 2 minutes ( ! ). When taking this away
> I found some improvement in the throughput numbers I was seeing. To more
> closely emulate a real NN we should improve this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
