[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[
https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290300#comment-15290300
]
Yiqun Lin commented on HDFS-10400:
--
{quote}
So I was wondering maybe the hdfs dfs -put was able to recover and finish
successfully.
{qupte}
Agree with [~knoguchi].
> hdfs dfs -put exits with zero on error
> --
>
> Key: HDFS-10400
> URL: https://issues.apache.org/jira/browse/HDFS-10400
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Jo Desmet
>Assignee: Yiqun Lin
> Attachments: HDFS-10400.001.patch, HDFS-10400.002.patch
>
>
> On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file
> that is big enough to go over the limit. As a result, the command fails with
> an exception, however the command terminates normally (exit code 0).
> Expectation is that any detectable failure generates an exit code different
> than zero.
> Documentation on
> https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states:
> Exit Code:
> Returns 0 on success and -1 on error.
> following is the exception generated:
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464)
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning
> BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114
> 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode
> DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[
https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290236#comment-15290236
]
Koji Noguchi commented on HDFS-10400:
-
My point was, the INFO message on the description
{noformat}
16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
java.io.EOFException: Premature EOF: no length prefix available
at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282)
at
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352)
at
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271)
at
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464)
16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning
BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114
16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode
DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK]
{noformat}
does not say the write failed. It says it just failed writing to one datanode.
So I was wondering maybe the hdfs dfs -put was able to recover and finish
successfully.
> hdfs dfs -put exits with zero on error
> --
>
> Key: HDFS-10400
> URL: https://issues.apache.org/jira/browse/HDFS-10400
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Jo Desmet
>Assignee: Yiqun Lin
> Attachments: HDFS-10400.001.patch, HDFS-10400.002.patch
>
>
> On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file
> that is big enough to go over the limit. As a result, the command fails with
> an exception, however the command terminates normally (exit code 0).
> Expectation is that any detectable failure generates an exit code different
> than zero.
> Documentation on
> https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states:
> Exit Code:
> Returns 0 on success and -1 on error.
> following is the exception generated:
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464)
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning
> BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114
> 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode
> DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[
https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290165#comment-15290165
]
Jo Desmet commented on HDFS-10400:
--
{quote}
Jo Desmet, how did you verify that dfs -put failed?
{quote}
Unix, simply using {{$?}}. I did confirm that I was able to detect other error
conditions, like source file non-existing. Additionally when the untrapped
exception occurred, I also verified that the file was either missing or
incomplete at the target location. [~linyiqun], checking checksum seems to be
extreme if an exception is somehow already generated?
> hdfs dfs -put exits with zero on error
> --
>
> Key: HDFS-10400
> URL: https://issues.apache.org/jira/browse/HDFS-10400
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Jo Desmet
>Assignee: Yiqun Lin
> Attachments: HDFS-10400.001.patch, HDFS-10400.002.patch
>
>
> On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file
> that is big enough to go over the limit. As a result, the command fails with
> an exception, however the command terminates normally (exit code 0).
> Expectation is that any detectable failure generates an exit code different
> than zero.
> Documentation on
> https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states:
> Exit Code:
> Returns 0 on success and -1 on error.
> following is the exception generated:
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464)
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning
> BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114
> 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode
> DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[
https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288127#comment-15288127
]
Yiqun Lin commented on HDFS-10400:
--
Hi, [~knoguchi], your commets looks right. I have tested the case and found
that the exception will also catch exception in {{FsShell}} and return -1 if
the {{Command}} not caught this.
So there is one possibility that the IOException in {{DataStreamer#run}} was
caught and not be threw out. Looks these code
{code}
} catch (Throwable e) {
// Log warning if there was a real error.
if (!errorState.isRestartingNode()) {
// Since their messages are descriptive enough, do not always
// log a verbose stack-trace WARN for quota exceptions.
if (e instanceof QuotaExceededException) {
LOG.debug("DataStreamer Quota Exception", e);
} else {
LOG.warn("DataStreamer Exception", e);
}
}
lastException.set(e);
assert !(e instanceof NullPointerException);
errorState.setInternalError();
if (!errorState.isNodeMarked()) {
// Not a datanode issue
streamerClosed = true;
}
}
{code}
Because the IOException was not threw out, and command will execute normally
and return code 0. But actually the Exception have happened in copying files.
If I am think correctly, we can do a file's checksum check between source and
destination file.
> hdfs dfs -put exits with zero on error
> --
>
> Key: HDFS-10400
> URL: https://issues.apache.org/jira/browse/HDFS-10400
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Jo Desmet
>Assignee: Yiqun Lin
> Attachments: HDFS-10400.001.patch, HDFS-10400.002.patch
>
>
> On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file
> that is big enough to go over the limit. As a result, the command fails with
> an exception, however the command terminates normally (exit code 0).
> Expectation is that any detectable failure generates an exit code different
> than zero.
> Documentation on
> https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states:
> Exit Code:
> Returns 0 on success and -1 on error.
> following is the exception generated:
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464)
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning
> BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114
> 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode
> DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[ https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286769#comment-15286769 ] Koji Noguchi commented on HDFS-10400: - I'm confused with this jira. Do we have a test case when hdfs dfs -put fails but still exits with zero? Uncaught exception on jvm would definitely exit with non-zero value. Also, the stack trace in the description is only "INFO", and usually it goes through a couple of retries before error-ing out with ERROR/FATAL. [~jo_des...@yahoo.com], how did you verify that dfs -put failed? > hdfs dfs -put exits with zero on error > -- > > Key: HDFS-10400 > URL: https://issues.apache.org/jira/browse/HDFS-10400 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jo Desmet >Assignee: Yiqun Lin > Attachments: HDFS-10400.001.patch, HDFS-10400.002.patch > > > On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file > that is big enough to go over the limit. As a result, the command fails with > an exception, however the command terminates normally (exit code 0). > Expectation is that any detectable failure generates an exit code different > than zero. > Documentation on > https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states: > Exit Code: > Returns 0 on success and -1 on error. > following is the exception generated: > 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream > java.io.EOFException: Premature EOF: no length prefix available > at > org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464) > 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning > BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114 > 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode > DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[
https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286212#comment-15286212
]
Hadoop QA commented on HDFS-10400:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 24s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 54s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} hadoop-common-project/hadoop-common: patch
generated 0 new + 85 unchanged - 1 fixed = 85 total (was 86) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 1s {color} |
{color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 16s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.net.TestDNS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804362/HDFS-10400.002.patch |
| JIRA Issue | HDFS-10400 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f90b077365f1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 9fe5828 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/15459/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HDFS-Build/15459/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/15459/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15459/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> hdfs dfs -put exits with zero on error
> --
>
> Key: HDFS-10400
> URL: https://issues.apache.org/jira/browse/HDFS-10400
>
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[
https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286034#comment-15286034
]
Hadoop QA commented on HDFS-10400:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m 8s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 24s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 0s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} hadoop-common-project/hadoop-common: patch
generated 0 new + 34 unchanged - 1 fixed = 34 total (was 35) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 20s {color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 10s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.TestFsShellReturnCode |
| | hadoop.fs.viewfs.TestViewFsTrash |
| | hadoop.fs.TestFsShellCopy |
| | hadoop.fs.TestTrash |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12804333/HDFS-10400.001.patch |
| JIRA Issue | HDFS-10400 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a2ea0bcc5dc5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 2c91fd8 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/15456/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HDFS-Build/15456/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/15456/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15456/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[
https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285969#comment-15285969
]
Yiqun Lin commented on HDFS-10400:
--
I looked into tha code again. I found the method {{displayError}} will increase
the {{numErrors}} count the IOException caught in inner method will not
influence the result. So it seems there was some un-trapped exception happened
here as [~jo_des...@yahoo.com] memtioned in comment. I would like to do
improvement in two palces.
* Caught un-trapped exception in {{Command#run}} method.
* The exit code for errors here is not same as instruction in
https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put. Here is
return the {{1}}.
I will post a patch later, thanks for review.
> hdfs dfs -put exits with zero on error
> --
>
> Key: HDFS-10400
> URL: https://issues.apache.org/jira/browse/HDFS-10400
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Jo Desmet
>Assignee: Yiqun Lin
>
> On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file
> that is big enough to go over the limit. As a result, the command fails with
> an exception, however the command terminates normally (exit code 0).
> Expectation is that any detectable failure generates an exit code different
> than zero.
> Documentation on
> https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states:
> Exit Code:
> Returns 0 on success and -1 on error.
> following is the exception generated:
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464)
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning
> BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114
> 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode
> DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[ https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284641#comment-15284641 ] Kihwal Lee commented on HDFS-10400: --- [~linyiqun], thanks for working in this. I assigned it to you. > hdfs dfs -put exits with zero on error > -- > > Key: HDFS-10400 > URL: https://issues.apache.org/jira/browse/HDFS-10400 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jo Desmet >Assignee: Yiqun Lin > > On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file > that is big enough to go over the limit. As a result, the command fails with > an exception, however the command terminates normally (exit code 0). > Expectation is that any detectable failure generates an exit code different > than zero. > Documentation on > https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states: > Exit Code: > Returns 0 on success and -1 on error. > following is the exception generated: > 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream > java.io.EOFException: Premature EOF: no length prefix available > at > org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464) > 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning > BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114 > 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode > DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[
https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283529#comment-15283529
]
Yiqun Lin commented on HDFS-10400:
--
I have looked into the code. When the src local files's num is more than one,
it will invoke its parent method. And the potenial IOException in method
{{processArgument}} is catched in {{Command}} and not be threw again.
{code}
protected void processArguments(LinkedList args)
throws IOException {
for (PathData arg : args) {
try {
processArgument(arg);
} catch (IOException e) {
displayError(e);
}
}
}
{code}
The similar case is also happens in {{Commands#processPaths}}. And these method
will be involed in {{processRawArguments(args);}}, its IOException will not be
thred here. The numErrors will also not be incrased.
{code}
public int run(String...argv) {
LinkedList args = new LinkedList(Arrays.asList(argv));
try {
if (isDeprecated()) {
displayWarning(
"DEPRECATED: Please use '"+ getReplacementCommand() + "' instead.");
}
processOptions(args);
processRawArguments(args);
} catch (CommandInterruptException e) {
displayError("Interrupted");
return 130;
} catch (IOException e) {
displayError(e);
}
return (numErrors == 0) ? exitCode : exitCodeForError();
}
{code}
So I think this is likely the reason.
I'm glad to do further work for this, who can assign this JIRA to me? It seems
that I can't assign JIRA to myself now, thanks.
> hdfs dfs -put exits with zero on error
> --
>
> Key: HDFS-10400
> URL: https://issues.apache.org/jira/browse/HDFS-10400
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Jo Desmet
>
> On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file
> that is big enough to go over the limit. As a result, the command fails with
> an exception, however the command terminates normally (exit code 0).
> Expectation is that any detectable failure generates an exit code different
> than zero.
> Documentation on
> https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states:
> Exit Code:
> Returns 0 on success and -1 on error.
> following is the exception generated:
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464)
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning
> BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114
> 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode
> DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[
https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283309#comment-15283309
]
Jo Desmet commented on HDFS-10400:
--
Hadoop version is 2.7.1.2.3.2.0-2950
Apologies, didn't notice that I used the hadoop 1 link.
I doubt that a fix will cause issues, as all other exceptions so far have been
returning a valid exit code (non-zero). Also filling up a filesystem will
hopefully be a rare event. In my case the event went undetected, and a lot of
wasteful processing continued.
It does look however that other exceptions are handled - things like a {{put}}
of a non existing file - as they do not generate a Java Exception but a clean
error report. I think what needs to happen is put a generic handler in place
for un-trapped exceptions, still do the exception dump, but then exit with
proper exit code.
There is precedent for this issue, please see
- [HADOOP-4340|https://issues.apache.org/jira/browse/hadoop-4340]
- [MAPREDUCE-3179|https://issues.apache.org/jira/browse/MAPREDUCE-3179]
> hdfs dfs -put exits with zero on error
> --
>
> Key: HDFS-10400
> URL: https://issues.apache.org/jira/browse/HDFS-10400
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Jo Desmet
>
> On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file
> that is big enough to go over the limit. As a result, the command fails with
> an exception, however the command terminates normally (exit code 0).
> Expectation is that any detectable failure generates an exit code different
> than zero.
> Documentation on
> https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states:
> Exit Code:
> Returns 0 on success and -1 on error.
> following is the exception generated:
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464)
> 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning
> BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114
> 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode
> DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error
[ https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283293#comment-15283293 ] Mingliang Liu commented on HDFS-10400: -- Thanks for reporting this. Which version are you working on? I saw you provided a hadoop 1 doc. I'm also concerned that the fix, if needed, may be backwards incompatible as some existing scripts may rely on this behavior, though it's makes less sense than you proposed. > hdfs dfs -put exits with zero on error > -- > > Key: HDFS-10400 > URL: https://issues.apache.org/jira/browse/HDFS-10400 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Jo Desmet > > On a filesystem that is about to fill up, execute "hdfs dfs -put" for a file > that is big enough to go over the limit. As a result, the command fails with > an exception, however the command terminates normally (exit code 0). > Expectation is that any detectable failure generates an exit code different > than zero. > Documentation on > https://hadoop.apache.org/docs/r1.2.1/file_system_shell.html#put states: > Exit Code: > Returns 0 on success and -1 on error. > following is the exception generated: > 16/05/11 13:37:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream > java.io.EOFException: Premature EOF: no length prefix available > at > org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2282) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1352) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1271) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:464) > 16/05/11 13:37:07 INFO hdfs.DFSClient: Abandoning > BP-1964113808-130.8.138.99-1446787670498:blk_1073835906_95114 > 16/05/11 13:37:08 INFO hdfs.DFSClient: Excluding datanode > DatanodeInfoWithStorage[130.8.138.99:50010,DS-eed7039a-8031-499e-85a5-7216b9d766a8,DISK] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
