[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[ https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687849#comment-13687849 ] Hudson commented on HADOOP-9582: Integrated in Hadoop-Yarn-trunk #245 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/245/]) HADOOP-9582. Non-existent file to hadoop fs -conf doesn't throw error. Contributed by Ashwin Shankar (Revision 1494331) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1494331 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShell.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShell.java Non-existent file to hadoop fs -conf doesn't throw error -- Key: HADOOP-9582 URL: https://issues.apache.org/jira/browse/HADOOP-9582 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha Reporter: Ashwin Shankar Assignee: Ashwin Shankar Fix For: 3.0.0, 0.23.9, 2.3.0 Attachments: HADOOP-9582-4-b23.txt, HADOOP-9582-4.txt, HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt When we run : hadoop fs -conf BAD_FILE -ls / we expect hadoop to throw an error,but it doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[ https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687932#comment-13687932 ] Hudson commented on HADOOP-9582: Integrated in Hadoop-Hdfs-0.23-Build #643 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/643/]) HADOOP-9582. Non-existent file to hadoop fs -conf doesn't throw error. Contributed by Ashwin Shankar (Revision 1494338) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1494338 Files : * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShell.java * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShell.java Non-existent file to hadoop fs -conf doesn't throw error -- Key: HADOOP-9582 URL: https://issues.apache.org/jira/browse/HADOOP-9582 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha Reporter: Ashwin Shankar Assignee: Ashwin Shankar Fix For: 3.0.0, 0.23.9, 2.3.0 Attachments: HADOOP-9582-4-b23.txt, HADOOP-9582-4.txt, HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt When we run : hadoop fs -conf BAD_FILE -ls / we expect hadoop to throw an error,but it doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[ https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687950#comment-13687950 ] Hudson commented on HADOOP-9582: Integrated in Hadoop-Hdfs-trunk #1435 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1435/]) HADOOP-9582. Non-existent file to hadoop fs -conf doesn't throw error. Contributed by Ashwin Shankar (Revision 1494331) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1494331 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShell.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShell.java Non-existent file to hadoop fs -conf doesn't throw error -- Key: HADOOP-9582 URL: https://issues.apache.org/jira/browse/HADOOP-9582 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha Reporter: Ashwin Shankar Assignee: Ashwin Shankar Fix For: 3.0.0, 0.23.9, 2.3.0 Attachments: HADOOP-9582-4-b23.txt, HADOOP-9582-4.txt, HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt When we run : hadoop fs -conf BAD_FILE -ls / we expect hadoop to throw an error,but it doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[ https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687995#comment-13687995 ] Hudson commented on HADOOP-9582: Integrated in Hadoop-Mapreduce-trunk #1462 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1462/]) HADOOP-9582. Non-existent file to hadoop fs -conf doesn't throw error. Contributed by Ashwin Shankar (Revision 1494331) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1494331 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShell.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShell.java Non-existent file to hadoop fs -conf doesn't throw error -- Key: HADOOP-9582 URL: https://issues.apache.org/jira/browse/HADOOP-9582 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha Reporter: Ashwin Shankar Assignee: Ashwin Shankar Fix For: 3.0.0, 0.23.9, 2.3.0 Attachments: HADOOP-9582-4-b23.txt, HADOOP-9582-4.txt, HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt When we run : hadoop fs -conf BAD_FILE -ls / we expect hadoop to throw an error,but it doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[ https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687022#comment-13687022 ] Jason Lowe commented on HADOOP-9582: bq. I did do this in one my previous patch,however Steve(on 22/May/13 06:01) recommended that raising AssertionFailedError would be better for diagnostics. Ah, yes, that makes sense for better diagnostics. I'm +1 on the latest patch and will commit later today. Non-existent file to hadoop fs -conf doesn't throw error -- Key: HADOOP-9582 URL: https://issues.apache.org/jira/browse/HADOOP-9582 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha Reporter: Ashwin Shankar Assignee: Ashwin Shankar Attachments: HADOOP-9582-4-b23.txt, HADOOP-9582-4.txt, HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt When we run : hadoop fs -conf BAD_FILE -ls / we expect hadoop to throw an error,but it doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[ https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687268#comment-13687268 ] Hudson commented on HADOOP-9582: Integrated in Hadoop-trunk-Commit #3970 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3970/]) HADOOP-9582. Non-existent file to hadoop fs -conf doesn't throw error. Contributed by Ashwin Shankar (Revision 1494331) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1494331 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShell.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShell.java Non-existent file to hadoop fs -conf doesn't throw error -- Key: HADOOP-9582 URL: https://issues.apache.org/jira/browse/HADOOP-9582 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha Reporter: Ashwin Shankar Assignee: Ashwin Shankar Attachments: HADOOP-9582-4-b23.txt, HADOOP-9582-4.txt, HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt When we run : hadoop fs -conf BAD_FILE -ls / we expect hadoop to throw an error,but it doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683829#comment-13683829
]
Ashwin Shankar commented on HADOOP-9582:
{quote}
TestGenericOptionsParser change now seems moot, since we didn't modify
GenericOptionsParser and it doesn't test the FsShell change.
{quote}
Yes,right. I've reverted that change in the latest patch.
{quote}
Nit: In TestFsShell rather than wielding AssertionFailedError directly we can
simply use assertTrue(th instanceof RuntimeException)
{quote}
I did do this in one my previous patch,however Steve(on 22/May/13 06:01)
recommended that raising AssertionFailedError would be better for diagnostics.
Should I still use assertTrue ?
Non-existent file to hadoop fs -conf doesn't throw error
--
Key: HADOOP-9582
URL: https://issues.apache.org/jira/browse/HADOOP-9582
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
Reporter: Ashwin Shankar
Assignee: Ashwin Shankar
Attachments: HADOOP-9582-4-b23.txt, HADOOP-9582-4.txt,
HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt
When we run :
hadoop fs -conf BAD_FILE -ls /
we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683845#comment-13683845
]
Hadoop QA commented on HADOOP-9582:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587893/HADOOP-9582-4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2647//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2647//console
This message is automatically generated.
Non-existent file to hadoop fs -conf doesn't throw error
--
Key: HADOOP-9582
URL: https://issues.apache.org/jira/browse/HADOOP-9582
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
Reporter: Ashwin Shankar
Assignee: Ashwin Shankar
Attachments: HADOOP-9582-4-b23.txt, HADOOP-9582-4.txt,
HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt
When we run :
hadoop fs -conf BAD_FILE -ls /
we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[ https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682845#comment-13682845 ] Jason Lowe commented on HADOOP-9582: Seems reasonable to at least make a small improvement to FsShell to catch bad -conf parameters, and that should be lower risk than having GenericOptionsParser start blindly setting all Configurations passed to it as non-quiet. Some comments on the latest patch: * TestGenericOptionsParser change now seems moot, since we didn't modify GenericOptionsParser and it doesn't test the FsShell change. * Nit: In TestFsShell rather than wielding AssertionFailedError directly we can simply use assertTrue(th instanceof RuntimeException) Non-existent file to hadoop fs -conf doesn't throw error -- Key: HADOOP-9582 URL: https://issues.apache.org/jira/browse/HADOOP-9582 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha Reporter: Ashwin Shankar Attachments: HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt When we run : hadoop fs -conf BAD_FILE -ls / we expect hadoop to throw an error,but it doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[ https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13675372#comment-13675372 ] Ashwin Shankar commented on HADOOP-9582: Hi Jason, Thanks for your comments. Yes this patch doesn't address the problem in other places apart from FsShell. But could we check this in so that we are good atleast from the FsShell standpoint. Maybe we could create another JIRA to investigate whether we break any use-case if we make the change directly in GenericOptionsParser. Non-existent file to hadoop fs -conf doesn't throw error -- Key: HADOOP-9582 URL: https://issues.apache.org/jira/browse/HADOOP-9582 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha Reporter: Ashwin Shankar Attachments: HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt When we run : hadoop fs -conf BAD_FILE -ls / we expect hadoop to throw an error,but it doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[ https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671683#comment-13671683 ] Jason Lowe commented on HADOOP-9582: This would solve the problem for hadoop fs but since -conf is a GenericOptionsParser thing, it doesn't address the many other places where -conf could be specified and processed by GenericOptionsParser. Is the real fix in a more central location like GenericOptionsParser or Configuration? If we just target FsShell then we'll have to address this behavior in many places and open the door to inconsistencies with missing configuration files based on who's using GenericOptionsParser. Non-existent file to hadoop fs -conf doesn't throw error -- Key: HADOOP-9582 URL: https://issues.apache.org/jira/browse/HADOOP-9582 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha Reporter: Ashwin Shankar Attachments: HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt When we run : hadoop fs -conf BAD_FILE -ls / we expect hadoop to throw an error,but it doesn't. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1341#comment-1341
]
Hadoop QA commented on HADOOP-9582:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584745/HADOOP-9582.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2569//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2569//console
This message is automatically generated.
Non-existent file to hadoop fs -conf doesn't throw error
--
Key: HADOOP-9582
URL: https://issues.apache.org/jira/browse/HADOOP-9582
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
Reporter: Ashwin Shankar
Attachments: HADOOP-9582.txt, HADOOP-9582.txt, HADOOP-9582.txt
When we run :
hadoop fs -conf BAD_FILE -ls /
we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665048#comment-13665048
]
Steve Loughran commented on HADOOP-9582:
Ashwin
# I don't know the history of that, let's ask.
# fs shell calls {{setQuietMode(true)}} in {{init()}} and doesn't provide a way
to turn it off. So even if we consider your current patch surplus, we do still
need a way to turn the flag on.
Non-existent file to hadoop fs -conf doesn't throw error
--
Key: HADOOP-9582
URL: https://issues.apache.org/jira/browse/HADOOP-9582
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
Reporter: Ashwin Shankar
Attachments: HADOOP-9582.txt, HADOOP-9582.txt
When we run :
hadoop fs -conf BAD_FILE -ls /
we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664005#comment-13664005
]
Steve Loughran commented on HADOOP-9582:
Looks good, both code and tests.
# there's a couple of places where an extra space or two between things would
comply more with the style guidelines
# the final {{assertTrue}} could be better on diagnostics if a different fault
is raised. A {{throw new AssertionFailedError(Expected FileNotFound, got:
+th).initCause(th);}} would do this.
Non-existent file to hadoop fs -conf doesn't throw error
--
Key: HADOOP-9582
URL: https://issues.apache.org/jira/browse/HADOOP-9582
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
Reporter: Ashwin Shankar
Attachments: HADOOP-9582.txt
When we run :
hadoop fs -conf BAD_FILE -ls /
we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664238#comment-13664238
]
Hadoop QA commented on HADOOP-9582:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584323/HADOOP-9582.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2557//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2557//console
This message is automatically generated.
Non-existent file to hadoop fs -conf doesn't throw error
--
Key: HADOOP-9582
URL: https://issues.apache.org/jira/browse/HADOOP-9582
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
Reporter: Ashwin Shankar
Attachments: HADOOP-9582.txt, HADOOP-9582.txt
When we run :
hadoop fs -conf BAD_FILE -ls /
we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664282#comment-13664282
]
Steve Loughran commented on HADOOP-9582:
I've been looking at this a bit more, and now I'm worried about some
compatibility issue. It looks like {{-conf}} can be used to specify any
resource path. Is this right, or am I misreading it?
If I am right, I could go {{--conf ftp://common/stdconf.xml}}, or {{--conf
http://service/cluster-site.xml}}.
This could be handy -so I'm worried that some people may already be using it.
If they are, we've got a problem.
At the same time, delving into {{Configuration.loadResource()}} looks like it
gets the URI of the {{Path}} instance, calls {{getPath()}} on it and then
converts it to a file for loading (skipping missing entries).
Is that right? That the {{Path}} representation of a command line path is
simply an intermediate state -and that it doesn't interfere with the
intermediate state.
If backwards compatibility isn't at risk -which I think is the case- then this
check is something we can keep in, though it may be safer to build the
{code}
resource = new Path(value)
File confFile = new File((resource).toUri().getPath()); //from Configuration
if (! confFile.exists()) { throw ... }
conf.addResource(resource);
{code}
The advantage with this approach is we can be confident the relative path
resolution logic is consistent in both places. Which is something that tests
could look for as well.
(I'm just adding extra work here, aren't I? Sorry)
Non-existent file to hadoop fs -conf doesn't throw error
--
Key: HADOOP-9582
URL: https://issues.apache.org/jira/browse/HADOOP-9582
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
Reporter: Ashwin Shankar
Attachments: HADOOP-9582.txt, HADOOP-9582.txt
When we run :
hadoop fs -conf BAD_FILE -ls /
we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664341#comment-13664341
]
Steve Loughran commented on HADOOP-9582:
My current (YARN-117) fork of Hadoop trunk has an extra test in Configuration
-it doesn't load empty files. This is the patch from HADOOP-9453, adding a
{{file.length() 0}} to the criteria, and stopping the XML parser failing on
an empty file.
Should your -conf code reject empty files too? Today that triggers the XML
parser failure -it's hard to see why we'd want to permit it once HADOOP-9453
gets checked in
Non-existent file to hadoop fs -conf doesn't throw error
--
Key: HADOOP-9582
URL: https://issues.apache.org/jira/browse/HADOOP-9582
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
Reporter: Ashwin Shankar
Attachments: HADOOP-9582.txt, HADOOP-9582.txt
When we run :
hadoop fs -conf BAD_FILE -ls /
we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to hadoop fs -conf doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13664558#comment-13664558
]
Ashwin Shankar commented on HADOOP-9582:
{quote}I've been looking at this a bit more, and now I'm worried about some
compatibility issue. It looks like -conf can be used to specify any resource
path. Is this right, or am I misreading it?{quote}
I'm not sure whether URIs are allowed for '-conf' option.I checked up the
documentation and javadoc, it doesn't talk about URIs. But I would tend to
believe that URIs are allowed ,since its allowed for other options like hadoop
fs -fs URI.If this is the case,then you're right,we have a problem.
{quote}At the same time, delving into Configuration.loadResource() looks like
it gets the URI of the Path instance, calls getPath() on it and then converts
it to a file for loading (skipping missing entries).{quote}
Yes,that's right. Here is where it gets interesting and I have a question for
you.
Configuration.loadResource() has the following code :
{code}
if (doc == null root == null) {
if (quiet)
return null;
throw new RuntimeException(resource + not found);
}
{code}
Looking at the code,it seems that if 'file' doesn't exist we do throw
RuntimeException if the 'quiet' flag is false,which means removing the 'quiet'
flag seems to solve our problem.My question is why is there a 'quiet' flag in
the first place ? I tried to look at svn file history and this particular code
snippet goes back to early days(2006),hence no documentation about it.
Do you know about this flag ?
Is there some ancient use-case where its legal to give a non-existent config
file as arguments while loading it 'quietly' ? because this patch will break
that.
Non-existent file to hadoop fs -conf doesn't throw error
--
Key: HADOOP-9582
URL: https://issues.apache.org/jira/browse/HADOOP-9582
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
Reporter: Ashwin Shankar
Attachments: HADOOP-9582.txt, HADOOP-9582.txt
When we run :
hadoop fs -conf BAD_FILE -ls /
we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
