[
https://issues.apache.org/jira/browse/HADOOP-17268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jinglun updated HADOOP-17268:
-----------------------------
Description:
My users recently complained 'The NameNode is much slower than before' to me.
The reason is the cluster and jobs are getting bigger and bigger. So is the
pressure of NameNode. I explained the pressure was heavy so the rpc requests
must wait, but they were not satisfied. Because they thought the original
quality of the service should be guaranteed. They were never told the NameNode
would be so slow and all their services were built based on the assumption that
the NameNode would always respond as fast as before.
From the user's standpoint they are right. So my question is how to give the
user a guarantee about RPC requests. The natural idea is RPC Quota, just like
name quota and space quota. The quota can help users to understand the rpc
requests are also a limit resource. And when they apply quota to the
administrator, the admin would have the chance to distribute the resource and
make a plan for the cluster. e.g. We have 200 quota for addBlock and they are
all allocated. Even the peak doesn't reach 200, I should reject other users
from applying to reserve the resource. The new user should be mounted to other
namespaces.
It's still an initial idea now. I'll think again carefully and make a detailed
proposal. All advice are welcome!
was:
My users recently complained 'The NameNode is much slower than before' to me.
The reason is the cluster and jobs are getting bigger and bigger. So is the
pressure of NameNode. I explained the pressure was heavy so the rpc requests
must wait, but they were not satisfied. Because they thought the original
quality of the service should be guaranteed. They were never told the NameNode
would be so slow and all their services were built based on the assumption that
the NameNode would always respond as fast as before.
>From the user's standpoint they are right. So my question is how to give the
>user a guarantee about RPC requests. The natural idea is RPC Quota, just like
>name quota and space quota. The quota can help users to understand the rpc
>requests are also a limit resource. And when they apply quota to the
>administrator, the admin would have the chance to distribute the resource and
>make a plan for the cluster. e.g. We have 200 quota for addBlock and they are
>all allocated. Even the peak doesn't reach 200, I should reject other users
>from applying to reserve the resource. The new user should be mounted to an
other namespace.
It's still an initial idea now. I'll think again carefully and make a detailed
proposal.
> Add RPC Quota to NameNode.
> --------------------------
>
> Key: HADOOP-17268
> URL: https://issues.apache.org/jira/browse/HADOOP-17268
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Jinglun
> Assignee: Jinglun
> Priority: Major
> Attachments: HADOOP-17268.001.patch
>
>
> My users recently complained 'The NameNode is much slower than before' to me.
> The reason is the cluster and jobs are getting bigger and bigger. So is the
> pressure of NameNode. I explained the pressure was heavy so the rpc requests
> must wait, but they were not satisfied. Because they thought the original
> quality of the service should be guaranteed. They were never told the
> NameNode would be so slow and all their services were built based on the
> assumption that the NameNode would always respond as fast as before.
> From the user's standpoint they are right. So my question is how to give the
> user a guarantee about RPC requests. The natural idea is RPC Quota, just like
> name quota and space quota. The quota can help users to understand the rpc
> requests are also a limit resource. And when they apply quota to the
> administrator, the admin would have the chance to distribute the resource and
> make a plan for the cluster. e.g. We have 200 quota for addBlock and they are
> all allocated. Even the peak doesn't reach 200, I should reject other users
> from applying to reserve the resource. The new user should be mounted to
> other namespaces.
> It's still an initial idea now. I'll think again carefully and make a
> detailed proposal. All advice are welcome!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
