[
https://issues.apache.org/jira/browse/HDFS-17845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Palanisamy updated HDFS-17845:
--------------------------------------
Description:
When a {{QuotaExceededException}} happens during {{{}addBlock{}}}, nameNode
throws back to client but {{DataStreamer}} client logs this as {{DEBUG}}
instead of {{WARN.}}
{{[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java#L857]}}
Hard to troubleshoot them.
Also, for any DataStreamer exception, we set it in lastException.set(e) and
close the stream afterward. This lastException thrown in all the scenarios
(when the stream is created with right datanode list). If any error during
stream then it retries and send back to client. But in case of
QuotaExceededException, the namenode didn’t return any datanode list, so the
output stream was never created and also we missed throwing the exception to
the client.
[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java#L862]
[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java#L876]
was:
When a {{QuotaExceededException}} happens during {{{}addBlock{}}}, nameNode
throws back to client but {{DataStreamer}} client logs this as {{DEBUG}}
instead of {{WARN.}}
{{[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java#L857]}}
Hard to troubleshoot them.
> DataStreamer QuotaExceededException Should Be WARN/ERROR, Not DEBUG
> -------------------------------------------------------------------
>
> Key: HDFS-17845
> URL: https://issues.apache.org/jira/browse/HDFS-17845
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Reporter: Karthik Palanisamy
> Priority: Major
>
> When a {{QuotaExceededException}} happens during {{{}addBlock{}}}, nameNode
> throws back to client but {{DataStreamer}} client logs this as {{DEBUG}}
> instead of {{WARN.}}
> {{[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java#L857]}}
> Hard to troubleshoot them.
> Also, for any DataStreamer exception, we set it in lastException.set(e) and
> close the stream afterward. This lastException thrown in all the scenarios
> (when the stream is created with right datanode list). If any error during
> stream then it retries and send back to client. But in case of
> QuotaExceededException, the namenode didn’t return any datanode list, so the
> output stream was never created and also we missed throwing the exception to
> the client.
>
> [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java#L862]
> [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java#L876]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
