[jira] [Commented] (HADOOP-8183) Stop using mapred.used.genericoptionsparser to avoid unnecessary warnings
[
https://issues.apache.org/jira/browse/HADOOP-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465479#comment-13465479
]
Harsh J commented on HADOOP-8183:
-
Thanks Jason. Nicer OOB experience :-)
Stop using mapred.used.genericoptionsparser to avoid unnecessary warnings
---
Key: HADOOP-8183
URL: https://issues.apache.org/jira/browse/HADOOP-8183
Project: Hadoop Common
Issue Type: Improvement
Components: util
Affects Versions: 0.23.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
Fix For: 2.0.0-alpha, 0.23.4
Attachments: HADOOP-8183.patch
Its about time we stopped the following from appearing in 0.23/trunk:
{code}
12/03/19 20:53:51 WARN conf.Configuration: mapred.used.genericoptionsparser
is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8776) Provide an option in test-patch that can enable / disable compiling native code
[ https://issues.apache.org/jira/browse/HADOOP-8776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465504#comment-13465504 ] Hemanth Yamijala commented on HADOOP-8776: -- Eli / Thomas, would it be possible for one of you to take a look and see if its fine ? This patch is helping a few of us develop patches from Mac. Provide an option in test-patch that can enable / disable compiling native code --- Key: HADOOP-8776 URL: https://issues.apache.org/jira/browse/HADOOP-8776 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0 Reporter: Hemanth Yamijala Assignee: Hemanth Yamijala Priority: Minor Attachments: HADOOP-8776.patch, HADOOP-8776.patch, HADOOP-8776.patch The test-patch script in Hadoop source runs a native compile with the patch. On platforms like MAC, there are issues with the native compile that make it difficult to use test-patch. This JIRA is to try and provide an option to make the native compilation optional. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8776) Provide an option in test-patch that can enable / disable compiling native code
[ https://issues.apache.org/jira/browse/HADOOP-8776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465511#comment-13465511 ] Tom White commented on HADOOP-8776: --- +1 I tried the modified test-patch on a Mac with the --disable-native flag and it worked fine. Provide an option in test-patch that can enable / disable compiling native code --- Key: HADOOP-8776 URL: https://issues.apache.org/jira/browse/HADOOP-8776 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0 Reporter: Hemanth Yamijala Assignee: Hemanth Yamijala Priority: Minor Attachments: HADOOP-8776.patch, HADOOP-8776.patch, HADOOP-8776.patch The test-patch script in Hadoop source runs a native compile with the patch. On platforms like MAC, there are issues with the native compile that make it difficult to use test-patch. This JIRA is to try and provide an option to make the native compilation optional. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8845) When looking for parent paths info, globStatus must filter out non-directory elements to avoid an AccessControlException
[
https://issues.apache.org/jira/browse/HADOOP-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465519#comment-13465519
]
Harsh J commented on HADOOP-8845:
-
Ping? Its a trivial fix to not lookup non-directories and I have tests attached
(similar to pClosure5, but that mkdired everything and couldn't run into
EXECUTE-less issues as described).
When looking for parent paths info, globStatus must filter out non-directory
elements to avoid an AccessControlException
Key: HADOOP-8845
URL: https://issues.apache.org/jira/browse/HADOOP-8845
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: Harsh J
Labels: glob
Attachments: HADOOP-8845.patch, HADOOP-8845.patch, HADOOP-8845.patch
A brief description from my colleague Stephen Fritz who helped discover it:
{code}
[root@node1 ~]# su - hdfs
-bash-4.1$ echo My Test Stringtestfile -- just a text file, for testing
below
-bash-4.1$ hadoop dfs -mkdir /tmp/testdir -- create a directory
-bash-4.1$ hadoop dfs -mkdir /tmp/testdir/1 -- create a subdirectory
-bash-4.1$ hadoop dfs -put testfile /tmp/testdir/1/testfile -- put the test
file in the subdirectory
-bash-4.1$ hadoop dfs -put testfile /tmp/testdir/testfile -- put the test
file in the directory
-bash-4.1$ hadoop dfs -lsr /tmp/testdir
drwxr-xr-x - hdfs hadoop 0 2012-09-25 06:52 /tmp/testdir/1
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/1/testfile
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/testfile
All files are where we expect them...OK, let's try reading
-bash-4.1$ hadoop dfs -cat /tmp/testdir/testfile
My Test String -- success!
-bash-4.1$ hadoop dfs -cat /tmp/testdir/1/testfile
My Test String -- success!
-bash-4.1$ hadoop dfs -cat /tmp/testdir/*/testfile
My Test String -- success!
Note that we used an '*' in the cat command, and it correctly found the
subdirectory '/tmp/testdir/1', and ignore the regular file
'/tmp/testdir/testfile'
-bash-4.1$ exit
logout
[root@node1 ~]# su - testuser -- lets try it as a different user:
[testuser@node1 ~]$ hadoop dfs -lsr /tmp/testdir
drwxr-xr-x - hdfs hadoop 0 2012-09-25 06:52 /tmp/testdir/1
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/1/testfile
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/testfile
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/testfile
My Test String -- good
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/1/testfile
My Test String -- so far so good
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/*/testfile
cat: org.apache.hadoop.security.AccessControlException: Permission denied:
user=testuser, access=EXECUTE,
inode=/tmp/testdir/testfile:hdfs:hadoop:-rw-r--r--
{code}
Essentially, we hit a ACE with access=EXECUTE on file /tmp/testdir/testfile
cause we tried to access the /tmp/testdir/testfile/testfile as a path. This
shouldn't happen, as the testfile is a file and not a path parent to be
looked up upon.
{code}
2012-09-25 07:24:27,406 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 8020, call getFileInfo(/tmp/testdir/testfile/testfile)
{code}
Surprisingly the superuser avoids hitting into the error, as a result of
bypassing permissions, but that can be looked up on another JIRA - if it is
fine to let it be like that or not.
This JIRA targets a client-sided fix to not cause such /path/file/dir or
/path/file/file kinda lookups.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8863) Eclipse plugin may not be working on Juno due to changes in it
Harsh J created HADOOP-8863: --- Summary: Eclipse plugin may not be working on Juno due to changes in it Key: HADOOP-8863 URL: https://issues.apache.org/jira/browse/HADOOP-8863 Project: Hadoop Common Issue Type: Bug Components: contrib/eclipse-plugin Affects Versions: 1.2.0 Reporter: Harsh J Assignee: Harsh J We need to debug/investigate why it is so. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested
[
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HADOOP-8589:
Attachment: HADOOP-8589.patch
Confirming - Applying Andrey's patch does solve the issue in all 3 cases (same
as above, all org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem
tests passed in each count).
I've rebased his patch for trunk as it didn't apply cleanly (line offset
issues). Lets have a jenkins run of the same again.
ViewFs tests fail when tests and home dirs are nested
-
Key: HADOOP-8589
URL: https://issues.apache.org/jira/browse/HADOOP-8589
Project: Hadoop Common
Issue Type: Bug
Components: fs, test
Affects Versions: 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
Attachments: HADOOP-8589.patch, HADOOP-8589.patch, HADOOP-8859.patch
TestFSMainOperationsLocalFileSystem fails in case when the test root
directory is under the user's home directory, and the user's home dir is
deeper than 2 levels from /. This happens with the default 1-node
installation of Jenkins.
This is the failure log:
{code}
org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as
dir; cannot create link here
at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
at org.apache.hadoop.fs.viewfs.InodeTree.init(InodeTree.java:334)
at
org.apache.hadoop.fs.viewfs.ViewFileSystem$1.init(ViewFileSystem.java:167)
at
org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
at
org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
at
org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
...
Standard Output
2012-07-11 22:07:20,239 INFO mortbay.log (Slf4jLog.java:info(67)) - Home dir
base /var/lib
{code}
The reason for the failure is that the code tries to mount links for both
/var and /var/lib, and it fails for the 2nd one as the /var is mounted
already.
The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7941) NoClassDefFoundError while running distcp/archive
[
https://issues.apache.org/jira/browse/HADOOP-7941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HADOOP-7941:
Fix Version/s: (was: 0.24.0)
NoClassDefFoundError while running distcp/archive
-
Key: HADOOP-7941
URL: https://issues.apache.org/jira/browse/HADOOP-7941
Project: Hadoop Common
Issue Type: Bug
Components: build
Affects Versions: 0.23.1
Reporter: Ramya Sunil
bin/hadoop distcp
{noformat}
Exception in thread main java.lang.NoClassDefFoundError:
org/apache/hadoop/tools/DistCp
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.tools.DistCp
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class: org.apache.hadoop.tools.DistCp. Program will
exit.
{noformat}
Same is the case while running 'bin/hadoop archive'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7996) change location of the native libraries to lib instead of lib/native
[ https://issues.apache.org/jira/browse/HADOOP-7996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465565#comment-13465565 ] Harsh J commented on HADOOP-7996: - Hi Roman/Eric, Is this still needed? change location of the native libraries to lib instead of lib/native Key: HADOOP-7996 URL: https://issues.apache.org/jira/browse/HADOOP-7996 Project: Hadoop Common Issue Type: Sub-task Components: build, conf, documentation, scripts Reporter: Roman Shaposhnik Assignee: Eric Yang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7996) change location of the native libraries to lib instead of lib/native
[ https://issues.apache.org/jira/browse/HADOOP-7996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-7996: Fix Version/s: (was: 0.24.0) change location of the native libraries to lib instead of lib/native Key: HADOOP-7996 URL: https://issues.apache.org/jira/browse/HADOOP-7996 Project: Hadoop Common Issue Type: Sub-task Components: build, conf, documentation, scripts Reporter: Roman Shaposhnik Assignee: Eric Yang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8183) Stop using mapred.used.genericoptionsparser to avoid unnecessary warnings
[
https://issues.apache.org/jira/browse/HADOOP-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465576#comment-13465576
]
Hudson commented on HADOOP-8183:
Integrated in Hadoop-Hdfs-0.23-Build #388 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/388/])
svn merge -c 1302598 FIXES: HADOOP-8183. Stop using
mapred.used.genericoptions.parser (harsh) (Revision 1391274)
Result = UNSTABLE
jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391274
Files :
*
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
Stop using mapred.used.genericoptionsparser to avoid unnecessary warnings
---
Key: HADOOP-8183
URL: https://issues.apache.org/jira/browse/HADOOP-8183
Project: Hadoop Common
Issue Type: Improvement
Components: util
Affects Versions: 0.23.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
Fix For: 2.0.0-alpha, 0.23.4
Attachments: HADOOP-8183.patch
Its about time we stopped the following from appearing in 0.23/trunk:
{code}
12/03/19 20:53:51 WARN conf.Configuration: mapred.used.genericoptionsparser
is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8623) hadoop jar command should respect HADOOP_OPTS
[ https://issues.apache.org/jira/browse/HADOOP-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465575#comment-13465575 ] Hudson commented on HADOOP-8623: Integrated in Hadoop-Hdfs-0.23-Build #388 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/388/]) svn merge -c 1366126 FIXES: HADOOP-8623. hadoop jar command should respect HADOOP_OPTS. Contributed by Steven Willis. (Revision 1391276) Result = UNSTABLE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391276 Files : * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop jar command should respect HADOOP_OPTS - Key: HADOOP-8623 URL: https://issues.apache.org/jira/browse/HADOOP-8623 Project: Hadoop Common Issue Type: Improvement Components: scripts Affects Versions: 0.23.1, 2.0.0-alpha Reporter: Steven Willis Assignee: Steven Willis Priority: Minor Fix For: 3.0.0, 2.0.2-alpha, 0.23.4 Attachments: HADOOP-8623.patch The jar command to the hadoop script should use any set HADOOP_OPTS and HADOOP_CLIENT_OPTS environment variables like all the other commands. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8850) Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown() sometimes throws NPEs
[ https://issues.apache.org/jira/browse/HADOOP-8850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan A. Veselovsky updated HADOOP-8850: --- Attachment: (was: HADOOP-8850-vs-trunk.patch) Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown() sometimes throws NPEs - Key: HADOOP-8850 URL: https://issues.apache.org/jira/browse/HADOOP-8850 Project: Hadoop Common Issue Type: Improvement Reporter: Ivan A. Veselovsky Priority: Minor Attachments: HADOOP-8850-vs-trunk-2.patch Recommended to add null-checking. Suggested patch is attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8850) Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown() sometimes throws NPEs
[ https://issues.apache.org/jira/browse/HADOOP-8850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan A. Veselovsky updated HADOOP-8850: --- Attachment: HADOOP-8850-vs-trunk-2.patch patch updated with proper formatting settings: indent size = 2, no tab characters used. Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown() sometimes throws NPEs - Key: HADOOP-8850 URL: https://issues.apache.org/jira/browse/HADOOP-8850 Project: Hadoop Common Issue Type: Improvement Reporter: Ivan A. Veselovsky Priority: Minor Attachments: HADOOP-8850-vs-trunk-2.patch Recommended to add null-checking. Suggested patch is attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8850) Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown() sometimes throws NPEs
[ https://issues.apache.org/jira/browse/HADOOP-8850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan A. Veselovsky updated HADOOP-8850: --- Status: Open (was: Patch Available) Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown() sometimes throws NPEs - Key: HADOOP-8850 URL: https://issues.apache.org/jira/browse/HADOOP-8850 Project: Hadoop Common Issue Type: Improvement Reporter: Ivan A. Veselovsky Priority: Minor Attachments: HADOOP-8850-vs-trunk-2.patch Recommended to add null-checking. Suggested patch is attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested
[
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465586#comment-13465586
]
Hadoop QA commented on HADOOP-8589:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12546983/HADOOP-8589.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.ha.TestZKFailoverController
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1540//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1540//console
This message is automatically generated.
ViewFs tests fail when tests and home dirs are nested
-
Key: HADOOP-8589
URL: https://issues.apache.org/jira/browse/HADOOP-8589
Project: Hadoop Common
Issue Type: Bug
Components: fs, test
Affects Versions: 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
Attachments: HADOOP-8589.patch, HADOOP-8589.patch, HADOOP-8859.patch
TestFSMainOperationsLocalFileSystem fails in case when the test root
directory is under the user's home directory, and the user's home dir is
deeper than 2 levels from /. This happens with the default 1-node
installation of Jenkins.
This is the failure log:
{code}
org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as
dir; cannot create link here
at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
at org.apache.hadoop.fs.viewfs.InodeTree.init(InodeTree.java:334)
at
org.apache.hadoop.fs.viewfs.ViewFileSystem$1.init(ViewFileSystem.java:167)
at
org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
at
org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
at
org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
...
Standard Output
2012-07-11 22:07:20,239 INFO mortbay.log (Slf4jLog.java:info(67)) - Home dir
base /var/lib
{code}
The reason for the failure is that the code tries to mount links for both
/var and /var/lib, and it fails for the 2nd one as the /var is mounted
already.
The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8551) fs -mkdir creates parent directories without the -p option
[ https://issues.apache.org/jira/browse/HADOOP-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465605#comment-13465605 ] Daryn Sharp commented on HADOOP-8551: - Our customers all updated their jobs. fs -mkdir creates parent directories without the -p option -- Key: HADOOP-8551 URL: https://issues.apache.org/jira/browse/HADOOP-8551 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0 Reporter: Robert Joseph Evans Assignee: John George Fix For: 0.23.3, 2.0.2-alpha Attachments: HADOOP-8551.patch, HADOOP-8551.patch, HADOOP-8551.patch, HADOOP-8551.patch hadoop fs -mkdir foo/bar will work even if bar is not present. It should only work if -p is given and foo is not present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8864) Addendum to HADOOP-8840: Add a coloring case for +0 results too.
Harsh J created HADOOP-8864: --- Summary: Addendum to HADOOP-8840: Add a coloring case for +0 results too. Key: HADOOP-8864 URL: https://issues.apache.org/jira/browse/HADOOP-8864 Project: Hadoop Common Issue Type: Bug Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8864.patch Noticed on MAPREDUCE-3223 that we failed to cover coloring the +0 case we print sometimes for doc-only patches. These can be colored green too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8864) Addendum to HADOOP-8840: Add a coloring case for +0 results too.
[ https://issues.apache.org/jira/browse/HADOOP-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-8864: Attachment: HADOOP-8864.patch Addendum to HADOOP-8840: Add a coloring case for +0 results too. Key: HADOOP-8864 URL: https://issues.apache.org/jira/browse/HADOOP-8864 Project: Hadoop Common Issue Type: Bug Reporter: Harsh J Assignee: Harsh J Attachments: HADOOP-8864.patch Noticed on MAPREDUCE-3223 that we failed to cover coloring the +0 case we print sometimes for doc-only patches. These can be colored green too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8864) Addendum to HADOOP-8840: Add a coloring case for +0 results too.
[ https://issues.apache.org/jira/browse/HADOOP-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-8864: Priority: Trivial (was: Major) Addendum to HADOOP-8840: Add a coloring case for +0 results too. Key: HADOOP-8864 URL: https://issues.apache.org/jira/browse/HADOOP-8864 Project: Hadoop Common Issue Type: Bug Reporter: Harsh J Assignee: Harsh J Priority: Trivial Attachments: HADOOP-8864.patch Noticed on MAPREDUCE-3223 that we failed to cover coloring the +0 case we print sometimes for doc-only patches. These can be colored green too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-4885) Try to restore failed replicas of Name Node storage (at checkpoint time)
[ https://issues.apache.org/jira/browse/HADOOP-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465633#comment-13465633 ] Bertrand Dechoux commented on HADOOP-4885: -- I did a grep -R dfs.name.dir.restore src on a downloaded version of Hadoop 1.0.3 and found no match. Maybe the fix version should be updated. Try to restore failed replicas of Name Node storage (at checkpoint time) Key: HADOOP-4885 URL: https://issues.apache.org/jira/browse/HADOOP-4885 Project: Hadoop Common Issue Type: Improvement Reporter: Boris Shkolnik Assignee: Boris Shkolnik Fix For: 1.0.3, 0.21.0 Attachments: HADOOP-4885-1.patch, HADOOP-4885-3.patch, HADOOP-4885-3.patch, HADOOP-4885.branch-1.patch, HADOOP-4885.branch-1.patch.2, HADOOP-4885.branch-1.patch.3, HADOOP-4885.patch, HADOOP-4885.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8864) Addendum to HADOOP-8840: Add a coloring case for +0 results too.
[ https://issues.apache.org/jira/browse/HADOOP-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465637#comment-13465637 ] Hudson commented on HADOOP-8864: Integrated in Hadoop-Hdfs-trunk-Commit #2844 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2844/]) HADOOP-8864. Addendum to HADOOP-8840: Add a coloring case for +0 results too. Contributed by Harsh J. (harsh) (Revision 1391480) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391480 Files : * /hadoop/common/trunk/dev-support/test-patch.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Addendum to HADOOP-8840: Add a coloring case for +0 results too. Key: HADOOP-8864 URL: https://issues.apache.org/jira/browse/HADOOP-8864 Project: Hadoop Common Issue Type: Improvement Reporter: Harsh J Assignee: Harsh J Priority: Trivial Fix For: 3.0.0 Attachments: HADOOP-8864.patch Noticed on MAPREDUCE-3223 that we failed to cover coloring the +0 case we print sometimes for doc-only patches. These can be colored green too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8864) Addendum to HADOOP-8840: Add a coloring case for +0 results too.
[ https://issues.apache.org/jira/browse/HADOOP-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465642#comment-13465642 ] Hudson commented on HADOOP-8864: Integrated in Hadoop-Common-trunk-Commit #2781 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2781/]) HADOOP-8864. Addendum to HADOOP-8840: Add a coloring case for +0 results too. Contributed by Harsh J. (harsh) (Revision 1391480) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391480 Files : * /hadoop/common/trunk/dev-support/test-patch.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Addendum to HADOOP-8840: Add a coloring case for +0 results too. Key: HADOOP-8864 URL: https://issues.apache.org/jira/browse/HADOOP-8864 Project: Hadoop Common Issue Type: Improvement Reporter: Harsh J Assignee: Harsh J Priority: Trivial Fix For: 3.0.0 Attachments: HADOOP-8864.patch Noticed on MAPREDUCE-3223 that we failed to cover coloring the +0 case we print sometimes for doc-only patches. These can be colored green too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8840) Fix the test-patch colorizer to cover all sorts of +1 lines.
[ https://issues.apache.org/jira/browse/HADOOP-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465641#comment-13465641 ] Hudson commented on HADOOP-8840: Integrated in Hadoop-Common-trunk-Commit #2781 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2781/]) HADOOP-8864. Addendum to HADOOP-8840: Add a coloring case for +0 results too. Contributed by Harsh J. (harsh) (Revision 1391480) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391480 Files : * /hadoop/common/trunk/dev-support/test-patch.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Fix the test-patch colorizer to cover all sorts of +1 lines. Key: HADOOP-8840 URL: https://issues.apache.org/jira/browse/HADOOP-8840 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Harsh J Assignee: Harsh J Fix For: 3.0.0 Attachments: HADOOP-8840.patch As noticed by Jason on HADOOP-8838, I missed some of the entries needed to be colorized. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8840) Fix the test-patch colorizer to cover all sorts of +1 lines.
[ https://issues.apache.org/jira/browse/HADOOP-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465658#comment-13465658 ] Hudson commented on HADOOP-8840: Integrated in Hadoop-Mapreduce-trunk-Commit #2803 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2803/]) HADOOP-8864. Addendum to HADOOP-8840: Add a coloring case for +0 results too. Contributed by Harsh J. (harsh) (Revision 1391480) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391480 Files : * /hadoop/common/trunk/dev-support/test-patch.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Fix the test-patch colorizer to cover all sorts of +1 lines. Key: HADOOP-8840 URL: https://issues.apache.org/jira/browse/HADOOP-8840 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Harsh J Assignee: Harsh J Fix For: 3.0.0 Attachments: HADOOP-8840.patch As noticed by Jason on HADOOP-8838, I missed some of the entries needed to be colorized. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8864) Addendum to HADOOP-8840: Add a coloring case for +0 results too.
[ https://issues.apache.org/jira/browse/HADOOP-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465659#comment-13465659 ] Hudson commented on HADOOP-8864: Integrated in Hadoop-Mapreduce-trunk-Commit #2803 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2803/]) HADOOP-8864. Addendum to HADOOP-8840: Add a coloring case for +0 results too. Contributed by Harsh J. (harsh) (Revision 1391480) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391480 Files : * /hadoop/common/trunk/dev-support/test-patch.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Addendum to HADOOP-8840: Add a coloring case for +0 results too. Key: HADOOP-8864 URL: https://issues.apache.org/jira/browse/HADOOP-8864 Project: Hadoop Common Issue Type: Improvement Reporter: Harsh J Assignee: Harsh J Priority: Trivial Fix For: 3.0.0 Attachments: HADOOP-8864.patch Noticed on MAPREDUCE-3223 that we failed to cover coloring the +0 case we print sometimes for doc-only patches. These can be colored green too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-4885) Try to restore failed replicas of Name Node storage (at checkpoint time)
[ https://issues.apache.org/jira/browse/HADOOP-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-4885: Fix Version/s: (was: 1.0.3) Try to restore failed replicas of Name Node storage (at checkpoint time) Key: HADOOP-4885 URL: https://issues.apache.org/jira/browse/HADOOP-4885 Project: Hadoop Common Issue Type: Improvement Reporter: Boris Shkolnik Assignee: Boris Shkolnik Fix For: 0.21.0 Attachments: HADOOP-4885-1.patch, HADOOP-4885-3.patch, HADOOP-4885-3.patch, HADOOP-4885.branch-1.patch, HADOOP-4885.branch-1.patch.2, HADOOP-4885.branch-1.patch.3, HADOOP-4885.patch, HADOOP-4885.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-4885) Try to restore failed replicas of Name Node storage (at checkpoint time)
[ https://issues.apache.org/jira/browse/HADOOP-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465661#comment-13465661 ] Harsh J commented on HADOOP-4885: - Lets visit HDFS-3075 for the backport. I removed the versioning from here as it was erroneous. Try to restore failed replicas of Name Node storage (at checkpoint time) Key: HADOOP-4885 URL: https://issues.apache.org/jira/browse/HADOOP-4885 Project: Hadoop Common Issue Type: Improvement Reporter: Boris Shkolnik Assignee: Boris Shkolnik Fix For: 0.21.0 Attachments: HADOOP-4885-1.patch, HADOOP-4885-3.patch, HADOOP-4885-3.patch, HADOOP-4885.branch-1.patch, HADOOP-4885.branch-1.patch.2, HADOOP-4885.branch-1.patch.3, HADOOP-4885.patch, HADOOP-4885.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8850) Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown() sometimes throws NPEs
[
https://issues.apache.org/jira/browse/HADOOP-8850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465686#comment-13465686
]
Hadoop QA commented on HADOOP-8850:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12546987/HADOOP-8850-vs-trunk-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
org.apache.hadoop.hdfs.TestPersistBlocks
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1541//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1541//console
This message is automatically generated.
Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown() sometimes throws
NPEs
-
Key: HADOOP-8850
URL: https://issues.apache.org/jira/browse/HADOOP-8850
Project: Hadoop Common
Issue Type: Improvement
Reporter: Ivan A. Veselovsky
Priority: Minor
Attachments: HADOOP-8850-vs-trunk-2.patch
Recommended to add null-checking.
Suggested patch is attached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-4885) Try to restore failed replicas of Name Node storage (at checkpoint time)
[
https://issues.apache.org/jira/browse/HADOOP-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465698#comment-13465698
]
Brandon Li commented on HADOOP-4885:
{quote}I did a grep -R dfs.name.dir.restore srcon a downloaded version of
Hadoop 1.0.3 and found no match.{quote}
The property name is dfs.namenode.name.dir.restore.
Try to restore failed replicas of Name Node storage (at checkpoint time)
Key: HADOOP-4885
URL: https://issues.apache.org/jira/browse/HADOOP-4885
Project: Hadoop Common
Issue Type: Improvement
Reporter: Boris Shkolnik
Assignee: Boris Shkolnik
Fix For: 0.21.0
Attachments: HADOOP-4885-1.patch, HADOOP-4885-3.patch,
HADOOP-4885-3.patch, HADOOP-4885.branch-1.patch,
HADOOP-4885.branch-1.patch.2, HADOOP-4885.branch-1.patch.3,
HADOOP-4885.patch, HADOOP-4885.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-4885) Try to restore failed replicas of Name Node storage (at checkpoint time)
[ https://issues.apache.org/jira/browse/HADOOP-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465706#comment-13465706 ] Bertrand Dechoux commented on HADOOP-4885: -- grep -R dfs.namenode.name.dir.restore * src/hdfs/org/apache/hadoop/hdfs/DFSConfigKeys.java: public static final String DFS_NAMENODE_NAME_DIR_RESTORE_KEY = dfs.namenode.name.dir.restore; Great! I will test it. The documentation does not seem updated but that's a detail. (same for the description of the jira...) Try to restore failed replicas of Name Node storage (at checkpoint time) Key: HADOOP-4885 URL: https://issues.apache.org/jira/browse/HADOOP-4885 Project: Hadoop Common Issue Type: Improvement Reporter: Boris Shkolnik Assignee: Boris Shkolnik Fix For: 0.21.0 Attachments: HADOOP-4885-1.patch, HADOOP-4885-3.patch, HADOOP-4885-3.patch, HADOOP-4885.branch-1.patch, HADOOP-4885.branch-1.patch.2, HADOOP-4885.branch-1.patch.3, HADOOP-4885.patch, HADOOP-4885.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-4885) Try to restore failed replicas of Name Node storage (at checkpoint time)
[ https://issues.apache.org/jira/browse/HADOOP-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bertrand Dechoux updated HADOOP-4885: - Release Note: Patch introduces new configuration switch dfs.name.dir.restore (boolean) enabling this functionality. Documentation needs to be updated. UPDATE : key was changed into dfs.namenode.name.dir.restore was:Patch introduces new configuration switch dfs.name.dir.restore (boolean) enabling this functionality. Documentation needs to be updated. Try to restore failed replicas of Name Node storage (at checkpoint time) Key: HADOOP-4885 URL: https://issues.apache.org/jira/browse/HADOOP-4885 Project: Hadoop Common Issue Type: Improvement Reporter: Boris Shkolnik Assignee: Boris Shkolnik Fix For: 0.21.0 Attachments: HADOOP-4885-1.patch, HADOOP-4885-3.patch, HADOOP-4885-3.patch, HADOOP-4885.branch-1.patch, HADOOP-4885.branch-1.patch.2, HADOOP-4885.branch-1.patch.3, HADOOP-4885.patch, HADOOP-4885.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8865) log warn when loading deprecated properties
[ https://issues.apache.org/jira/browse/HADOOP-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianbin Wei updated HADOOP-8865: Component/s: conf Affects Version/s: 3.0.0 Fix Version/s: 3.0.0 log warn when loading deprecated properties --- Key: HADOOP-8865 URL: https://issues.apache.org/jira/browse/HADOOP-8865 Project: Hadoop Common Issue Type: Improvement Components: conf Affects Versions: 3.0.0 Reporter: Jianbin Wei Fix For: 3.0.0 We need to log warn when loading deprecated properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8865) log warn when loading deprecated properties
Jianbin Wei created HADOOP-8865: --- Summary: log warn when loading deprecated properties Key: HADOOP-8865 URL: https://issues.apache.org/jira/browse/HADOOP-8865 Project: Hadoop Common Issue Type: Improvement Reporter: Jianbin Wei We need to log warn when loading deprecated properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8865) log warn when loading deprecated properties
[ https://issues.apache.org/jira/browse/HADOOP-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465717#comment-13465717 ] Jianbin Wei commented on HADOOP-8865: - Such log warn can (1) avoid potential use of deprecated properties in code (https://issues.apache.org/jira/browse/MAPREDUCE-4692) (2) speedup the deprecation for further improvement in the future log warn when loading deprecated properties --- Key: HADOOP-8865 URL: https://issues.apache.org/jira/browse/HADOOP-8865 Project: Hadoop Common Issue Type: Improvement Components: conf Affects Versions: 3.0.0 Reporter: Jianbin Wei Fix For: 3.0.0 We need to log warn when loading deprecated properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-4885) Try to restore failed replicas of Name Node storage (at checkpoint time)
[ https://issues.apache.org/jira/browse/HADOOP-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-4885: Release Note: Patch introduces new configuration switch dfs.name.dir.restore (boolean) enabling this functionality. Documentation needs to be updated. UPDATE: Config key is now dfs.namenode.name.dir.restore for 1.x and 2.x+ versions of HDFS was: Patch introduces new configuration switch dfs.name.dir.restore (boolean) enabling this functionality. Documentation needs to be updated. UPDATE : key was changed into dfs.namenode.name.dir.restore Try to restore failed replicas of Name Node storage (at checkpoint time) Key: HADOOP-4885 URL: https://issues.apache.org/jira/browse/HADOOP-4885 Project: Hadoop Common Issue Type: Improvement Reporter: Boris Shkolnik Assignee: Boris Shkolnik Fix For: 0.21.0 Attachments: HADOOP-4885-1.patch, HADOOP-4885-3.patch, HADOOP-4885-3.patch, HADOOP-4885.branch-1.patch, HADOOP-4885.branch-1.patch.2, HADOOP-4885.branch-1.patch.3, HADOOP-4885.patch, HADOOP-4885.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8865) log warn when loading deprecated properties
[ https://issues.apache.org/jira/browse/HADOOP-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465736#comment-13465736 ] Harsh J commented on HADOOP-8865: - Hi Jianbin, If am not mistaken, Configuration class already does log warnings when a deprecated property string is used. Are you talking of something else? log warn when loading deprecated properties --- Key: HADOOP-8865 URL: https://issues.apache.org/jira/browse/HADOOP-8865 Project: Hadoop Common Issue Type: Improvement Components: conf Affects Versions: 3.0.0 Reporter: Jianbin Wei Fix For: 3.0.0 We need to log warn when loading deprecated properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8865) log warn when loading deprecated properties
[
https://issues.apache.org/jira/browse/HADOOP-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465742#comment-13465742
]
Jianbin Wei commented on HADOOP-8865:
-
Yes I saw the code and it intends to do that only when the properties is being
set but not being loaded.
Following is some log after my initial change
{{[ ../hadoop-3.0.0-SNAPSHOT] ⚡ sbin/start-dfs.sh
12/09/28 09:54:03 WARN (Configuration.java:warnOnceIfDeprecated(826))
conf.Configuration: io.bytes.per.checksum is deprecated. Instead, use
dfs.bytes-per-checksum in core-default.xml
12/09/28 09:54:03 WARN (Configuration.java:warnOnceIfDeprecated(826))
conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS in
core-default.xml
12/09/28 09:54:03 WARN (Configuration.java:warnOnceIfDeprecated(826))
conf.Configuration: io.bytes.per.checksum is deprecated. Instead, use
dfs.bytes-per-checksum in core-default.xml
12/09/28 09:54:03 WARN (Configuration.java:warnOnceIfDeprecated(826))
conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS in
core-default.xml
12/09/28 09:54:03 WARN (Configuration.java:warnOnceIfDeprecated(826))
conf.Configuration: io.bytes.per.checksum is deprecated. Instead, use
dfs.bytes-per-checksum in core-default.xml}}
log warn when loading deprecated properties
---
Key: HADOOP-8865
URL: https://issues.apache.org/jira/browse/HADOOP-8865
Project: Hadoop Common
Issue Type: Improvement
Components: conf
Affects Versions: 3.0.0
Reporter: Jianbin Wei
Fix For: 3.0.0
We need to log warn when loading deprecated properties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-6311) Add support for unix domain sockets to JNI libs
[ https://issues.apache.org/jira/browse/HADOOP-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465753#comment-13465753 ] Colin Patrick McCabe commented on HADOOP-6311: -- This one has no code derived from Android. Add support for unix domain sockets to JNI libs --- Key: HADOOP-6311 URL: https://issues.apache.org/jira/browse/HADOOP-6311 Project: Hadoop Common Issue Type: New Feature Components: native Affects Versions: 0.20.0 Reporter: Todd Lipcon Assignee: Colin Patrick McCabe Attachments: 6311-trunk-inprogress.txt, HADOOP-6311.014.patch, HADOOP-6311.016.patch, HADOOP-6311.018.patch, HADOOP-6311.020.patch, HADOOP-6311-0.patch, HADOOP-6311-1.patch, hadoop-6311.txt For HDFS-347 we need to use unix domain sockets. This JIRA is to include a library in common which adds a o.a.h.net.unix package based on the code from Android (apache 2 license) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-6311) Add support for unix domain sockets to JNI libs
[ https://issues.apache.org/jira/browse/HADOOP-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HADOOP-6311: - Attachment: HADOOP-6311.020b.patch * Fix JavaDoc warnings Add support for unix domain sockets to JNI libs --- Key: HADOOP-6311 URL: https://issues.apache.org/jira/browse/HADOOP-6311 Project: Hadoop Common Issue Type: New Feature Components: native Affects Versions: 0.20.0 Reporter: Todd Lipcon Assignee: Colin Patrick McCabe Attachments: 6311-trunk-inprogress.txt, HADOOP-6311.014.patch, HADOOP-6311.016.patch, HADOOP-6311.018.patch, HADOOP-6311.020b.patch, HADOOP-6311.020.patch, HADOOP-6311-0.patch, HADOOP-6311-1.patch, hadoop-6311.txt For HDFS-347 we need to use unix domain sockets. This JIRA is to include a library in common which adds a o.a.h.net.unix package based on the code from Android (apache 2 license) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8753) LocalDirAllocator throws ArithmeticException: / by zero when there is no available space on configured local dir
[ https://issues.apache.org/jira/browse/HADOOP-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465765#comment-13465765 ] Benoy Antony commented on HADOOP-8753: -- I don't see an easy approach to test this. Let me know if anyone can think of anything to limit the size of a test directory. LocalDirAllocator throws ArithmeticException: / by zero when there is no available space on configured local dir -- Key: HADOOP-8753 URL: https://issues.apache.org/jira/browse/HADOOP-8753 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.2-alpha Reporter: Nishan Shetty Assignee: Benoy Antony Priority: Minor Attachments: HADOOP-8753.1.patch, YARN-16.patch 12/08/09 13:59:49 INFO mapreduce.Job: Task Id : attempt_1344492468506_0023_m_00_0, Status : FAILED java.lang.ArithmeticException: / by zero at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:371) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115) at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLocalPathForWrite(LocalDirsHandlerService.java:257) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:849) Instead of throwing exception directly we can log a warning saying no available space on configured local dir -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-6311) Add support for unix domain sockets to JNI libs
[
https://issues.apache.org/jira/browse/HADOOP-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465799#comment-13465799
]
Hadoop QA commented on HADOOP-6311:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12547013/HADOOP-6311.020b.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.ha.TestZKFailoverController
org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1542//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1542//console
This message is automatically generated.
Add support for unix domain sockets to JNI libs
---
Key: HADOOP-6311
URL: https://issues.apache.org/jira/browse/HADOOP-6311
Project: Hadoop Common
Issue Type: New Feature
Components: native
Affects Versions: 0.20.0
Reporter: Todd Lipcon
Assignee: Colin Patrick McCabe
Attachments: 6311-trunk-inprogress.txt, HADOOP-6311.014.patch,
HADOOP-6311.016.patch, HADOOP-6311.018.patch, HADOOP-6311.020b.patch,
HADOOP-6311.020.patch, HADOOP-6311-0.patch, HADOOP-6311-1.patch,
hadoop-6311.txt
For HDFS-347 we need to use unix domain sockets. This JIRA is to include a
library in common which adds a o.a.h.net.unix package based on the code from
Android (apache 2 license)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8386) hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu)
[ https://issues.apache.org/jira/browse/HADOOP-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465921#comment-13465921 ] Christopher Berner commented on HADOOP-8386: I didn't run into any other scripts that had this issue, but yes it would need to be handled everywhere as far as I can tell. I think another option (besides explicitly ignoring the output) would be to just add unset CDPATH at the beginning of the script. I've used that in scripts that had issues with environment variables before, and it works just fine. hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu) Key: HADOOP-8386 URL: https://issues.apache.org/jira/browse/HADOOP-8386 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 1.0.2 Environment: Ubuntu Reporter: Christopher Berner Attachments: hadoop.diff if the 'hadoop' script is run as 'bin/hadoop' on a distro where the 'cd' command prints to stdout, the script will fail due to this line: 'bin=`cd $bin; pwd`' Workaround: execute from the bin/ directory as './hadoop' Fix: change that line to 'bin=`cd $bin /dev/null; pwd`' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8863) Eclipse plugin may not be working on Juno due to changes in it
[ https://issues.apache.org/jira/browse/HADOOP-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465953#comment-13465953 ] Adam Berry commented on HADOOP-8863: fyi; http://mail-archives.apache.org/mod_mbox/hadoop-general/201209.mbox/browser is a thread talking about spinning these tools out. I can take a look at this to see if there is something easy to make the existing work. One question though, when you say juno, do you mean eclipse 3.8 or 4.2? Look in the about if you aren't sure. Eclipse plugin may not be working on Juno due to changes in it -- Key: HADOOP-8863 URL: https://issues.apache.org/jira/browse/HADOOP-8863 Project: Hadoop Common Issue Type: Bug Components: contrib/eclipse-plugin Affects Versions: 1.2.0 Reporter: Harsh J Assignee: Harsh J We need to debug/investigate why it is so. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8386) hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu)
[
https://issues.apache.org/jira/browse/HADOOP-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465962#comment-13465962
]
Andy Isaacson commented on HADOOP-8386:
---
The Gnu Autoconf manual has this to say:
http://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Special-Shell-Variables.html
{quote}
CDPATH
When this variable is set it specifies a list of directories to search when
invoking cd with a relative file name that did not start with ‘./’ or ‘../’.
Posix 1003.1-2001 says that if a nonempty directory name from CDPATH is used
successfully, cd prints the resulting absolute file name. Unfortunately this
output can break idioms like ‘abs=`cd src pwd`’ because abs receives the
name twice. Also, many shells do not conform to this part of Posix; for
example, zsh prints the result only if a directory name other than . was chosen
from CDPATH.
In practice the shells that have this problem also support unset, so you
can work around the problem as follows:
(unset CDPATH) /dev/null 21 unset CDPATH
You can also avoid output by ensuring that your directory name is absolute
or anchored at ‘./’, as in ‘abs=`cd ./src pwd`’.
{quote}
So the Bash behavior is specified by Posix, alas. It is specified to write to
stdout not stderr, so the patch is correct in that regard (I was concerned we
might also need {{21}} or similar).
So, LGTM. I'll upload a patch that conforms to test-patch expectations. Thanks
for the contribution, Christopher!
hadoop script doesn't work if 'cd' prints to stdout (default behavior in
Ubuntu)
Key: HADOOP-8386
URL: https://issues.apache.org/jira/browse/HADOOP-8386
Project: Hadoop Common
Issue Type: Bug
Components: scripts
Affects Versions: 1.0.2
Environment: Ubuntu
Reporter: Christopher Berner
Attachments: hadoop.diff
if the 'hadoop' script is run as 'bin/hadoop' on a distro where the 'cd'
command prints to stdout, the script will fail due to this line: 'bin=`cd
$bin; pwd`'
Workaround: execute from the bin/ directory as './hadoop'
Fix: change that line to 'bin=`cd $bin /dev/null; pwd`'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8386) hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu)
[ https://issues.apache.org/jira/browse/HADOOP-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Isaacson updated HADOOP-8386: -- Attachment: hadoop-8386.diff Uploading version of Christopher's patch that test-patch can apply. hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu) Key: HADOOP-8386 URL: https://issues.apache.org/jira/browse/HADOOP-8386 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 1.0.2 Environment: Ubuntu Reporter: Christopher Berner Attachments: hadoop-8386.diff, hadoop.diff if the 'hadoop' script is run as 'bin/hadoop' on a distro where the 'cd' command prints to stdout, the script will fail due to this line: 'bin=`cd $bin; pwd`' Workaround: execute from the bin/ directory as './hadoop' Fix: change that line to 'bin=`cd $bin /dev/null; pwd`' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8386) hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu)
[
https://issues.apache.org/jira/browse/HADOOP-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465969#comment-13465969
]
Hadoop QA commented on HADOOP-8386:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12547060/hadoop-8386.diff
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1543//console
This message is automatically generated.
hadoop script doesn't work if 'cd' prints to stdout (default behavior in
Ubuntu)
Key: HADOOP-8386
URL: https://issues.apache.org/jira/browse/HADOOP-8386
Project: Hadoop Common
Issue Type: Bug
Components: scripts
Affects Versions: 1.0.2
Environment: Ubuntu
Reporter: Christopher Berner
Attachments: hadoop-8386.diff, hadoop.diff
if the 'hadoop' script is run as 'bin/hadoop' on a distro where the 'cd'
command prints to stdout, the script will fail due to this line: 'bin=`cd
$bin; pwd`'
Workaround: execute from the bin/ directory as './hadoop'
Fix: change that line to 'bin=`cd $bin /dev/null; pwd`'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8852) DelegationTokenRenewer should be Singleton
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-8852: - Description: Updated description: DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal. Original issue: HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed. was:HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed. Summary: DelegationTokenRenewer should be Singleton (was: DelegationTokenRenewer thread is not stopped when its filesystem is closed) DelegationTokenRenewer should be Singleton -- Key: HADOOP-8852 URL: https://issues.apache.org/jira/browse/HADOOP-8852 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Tom White Assignee: Karthik Kambatla Attachments: hadoop-8852.patch, hadoop-8852.patch, hadoop-8852-v1.patch Updated description: DelegationTokenRenewer should be Singleton - the instance and renewer threads should be created/started lazily. The filesystems using the renewer shouldn't need to explicity start/stop the renewer, and only register/de-register for token renewal. Original issue: HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8386) hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu)
[ https://issues.apache.org/jira/browse/HADOOP-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Isaacson updated HADOOP-8386: -- Attachment: hadoop-8386-1.diff Let's try that again, against trunk rather than branch-1 this time. hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu) Key: HADOOP-8386 URL: https://issues.apache.org/jira/browse/HADOOP-8386 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 1.0.2 Environment: Ubuntu Reporter: Christopher Berner Attachments: hadoop-8386-1.diff, hadoop-8386.diff, hadoop.diff if the 'hadoop' script is run as 'bin/hadoop' on a distro where the 'cd' command prints to stdout, the script will fail due to this line: 'bin=`cd $bin; pwd`' Workaround: execute from the bin/ directory as './hadoop' Fix: change that line to 'bin=`cd $bin /dev/null; pwd`' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8756) Fix SEGV when libsnappy is in java.library.path but not LD_LIBRARY_PATH
[
https://issues.apache.org/jira/browse/HADOOP-8756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465980#comment-13465980
]
Andy Isaacson commented on HADOOP-8756:
---
+1
I've reviewed HADOOP-8756.004.patch and I have no further issues. This is a
good localized fix that is not dependent on nor obsoleted by the other patches
under discussion or checked in. Without this patch a local mvn -Pnative,dist
clean package -Dmaven.javadoc.skip=true -DskipTests; mvn test -Dtest=TestCodec
fails.
{code}
Running org.apache.hadoop.io.compress.TestCodec
Tests run: 21, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 15.956 sec
FAILURE!
Results :
Failed tests: testSnappyCodec(org.apache.hadoop.io.compress.TestCodec):
Snappy native available but Hadoop native not
{code}
Fix SEGV when libsnappy is in java.library.path but not LD_LIBRARY_PATH
---
Key: HADOOP-8756
URL: https://issues.apache.org/jira/browse/HADOOP-8756
Project: Hadoop Common
Issue Type: Bug
Components: native
Affects Versions: 2.0.2-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
Attachments: HADOOP-8756.002.patch, HADOOP-8756.003.patch,
HADOOP-8756.004.patch
We use {{System.loadLibrary(snappy)}} from the Java side. However in
libhadoop, we use {{dlopen}} to open libsnappy.so dynamically.
System.loadLibrary uses {{java.library.path}} to resolve libraries, and
{{dlopen}} uses {{LD_LIBRARY_PATH}} and the system paths to resolve
libraries. Because of this, the two library loading functions can be at odds.
We should fix this so we only load the library once, preferably using the
standard Java {{java.library.path}}.
We should also log the search path(s) we use for {{libsnappy.so}} when
loading fails, so that it's easier to diagnose configuration issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8386) hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu)
[
https://issues.apache.org/jira/browse/HADOOP-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466016#comment-13466016
]
Hadoop QA commented on HADOOP-8386:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12547070/hadoop-8386-1.diff
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.ha.TestZKFailoverController
org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1544//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1544//console
This message is automatically generated.
hadoop script doesn't work if 'cd' prints to stdout (default behavior in
Ubuntu)
Key: HADOOP-8386
URL: https://issues.apache.org/jira/browse/HADOOP-8386
Project: Hadoop Common
Issue Type: Bug
Components: scripts
Affects Versions: 1.0.2
Environment: Ubuntu
Reporter: Christopher Berner
Attachments: hadoop-8386-1.diff, hadoop-8386.diff, hadoop.diff
if the 'hadoop' script is run as 'bin/hadoop' on a distro where the 'cd'
command prints to stdout, the script will fail due to this line: 'bin=`cd
$bin; pwd`'
Workaround: execute from the bin/ directory as './hadoop'
Fix: change that line to 'bin=`cd $bin /dev/null; pwd`'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8551) fs -mkdir creates parent directories without the -p option
[ https://issues.apache.org/jira/browse/HADOOP-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466035#comment-13466035 ] Eli Collins commented on HADOOP-8551: - And I guess those jobs will never run on older 205 clusters. We have to worry about jobs (and tests) that run on multiple versions and didn't want to require people update their jobs (most of the other shell changes that were incompatible didn't actually require people update their jobs). fs -mkdir creates parent directories without the -p option -- Key: HADOOP-8551 URL: https://issues.apache.org/jira/browse/HADOOP-8551 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 0.23.3, 2.0.0-alpha, 3.0.0 Reporter: Robert Joseph Evans Assignee: John George Fix For: 0.23.3, 2.0.2-alpha Attachments: HADOOP-8551.patch, HADOOP-8551.patch, HADOOP-8551.patch, HADOOP-8551.patch hadoop fs -mkdir foo/bar will work even if bar is not present. It should only work if -p is given and foo is not present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8839) test-patch's -1 on @author tag presence doesn't cause a -1 to the overall result
[ https://issues.apache.org/jira/browse/HADOOP-8839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466050#comment-13466050 ] Aaron T. Myers commented on HADOOP-8839: +1, the patch looks good to me. test-patch's -1 on @author tag presence doesn't cause a -1 to the overall result Key: HADOOP-8839 URL: https://issues.apache.org/jira/browse/HADOOP-8839 Project: Hadoop Common Issue Type: Bug Components: build Reporter: Harsh J Assignee: Harsh J Priority: Trivial Attachments: HADOOP-8839.patch As observed on HADOOP-8838. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8386) hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu)
[ https://issues.apache.org/jira/browse/HADOOP-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466053#comment-13466053 ] Christopher Berner commented on HADOOP-8386: Awesome, thanks Andy! Glad to see this fixed :) hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu) Key: HADOOP-8386 URL: https://issues.apache.org/jira/browse/HADOOP-8386 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 1.0.2 Environment: Ubuntu Reporter: Christopher Berner Attachments: hadoop-8386-1.diff, hadoop-8386.diff, hadoop.diff if the 'hadoop' script is run as 'bin/hadoop' on a distro where the 'cd' command prints to stdout, the script will fail due to this line: 'bin=`cd $bin; pwd`' Workaround: execute from the bin/ directory as './hadoop' Fix: change that line to 'bin=`cd $bin /dev/null; pwd`' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Moved] (HADOOP-8866) SampleQuantiles#query is O(N^2) instead of O(N)
[ https://issues.apache.org/jira/browse/HADOOP-8866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers moved HDFS-3976 to HADOOP-8866: -- Target Version/s: 2.0.3-alpha (was: 2.0.3-alpha) Affects Version/s: (was: 2.0.3-alpha) 2.0.2-alpha Key: HADOOP-8866 (was: HDFS-3976) Project: Hadoop Common (was: Hadoop HDFS) SampleQuantiles#query is O(N^2) instead of O(N) --- Key: HADOOP-8866 URL: https://issues.apache.org/jira/browse/HADOOP-8866 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.2-alpha Reporter: Andrew Wang Assignee: Andrew Wang Priority: Minor Attachments: hdfs-3976-1.patch SampleQuantiles#query() does O(N) calls LinkedList#get() in a loop, rather than using an iterator. This makes query O(N^2), rather than O(N). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8866) SampleQuantiles#query is O(N^2) instead of O(N)
[ https://issues.apache.org/jira/browse/HADOOP-8866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466061#comment-13466061 ] Hudson commented on HADOOP-8866: Integrated in Hadoop-Mapreduce-trunk-Commit #2809 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2809/]) HADOOP-8866. SampleQuantiles#query is O(N^2) instead of O(N). Contributed by Andrew Wang. (Revision 1391711) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391711 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/SampleQuantiles.java SampleQuantiles#query is O(N^2) instead of O(N) --- Key: HADOOP-8866 URL: https://issues.apache.org/jira/browse/HADOOP-8866 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.2-alpha Reporter: Andrew Wang Assignee: Andrew Wang Priority: Minor Fix For: 2.0.3-alpha Attachments: hdfs-3976-1.patch SampleQuantiles#query() does O(N) calls LinkedList#get() in a loop, rather than using an iterator. This makes query O(N^2), rather than O(N). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8866) SampleQuantiles#query is O(N^2) instead of O(N)
[ https://issues.apache.org/jira/browse/HADOOP-8866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466064#comment-13466064 ] Hudson commented on HADOOP-8866: Integrated in Hadoop-Common-trunk-Commit #2787 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2787/]) HADOOP-8866. SampleQuantiles#query is O(N^2) instead of O(N). Contributed by Andrew Wang. (Revision 1391711) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391711 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/SampleQuantiles.java SampleQuantiles#query is O(N^2) instead of O(N) --- Key: HADOOP-8866 URL: https://issues.apache.org/jira/browse/HADOOP-8866 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.2-alpha Reporter: Andrew Wang Assignee: Andrew Wang Priority: Minor Fix For: 2.0.3-alpha Attachments: hdfs-3976-1.patch SampleQuantiles#query() does O(N) calls LinkedList#get() in a loop, rather than using an iterator. This makes query O(N^2), rather than O(N). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8866) SampleQuantiles#query is O(N^2) instead of O(N)
[ https://issues.apache.org/jira/browse/HADOOP-8866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466066#comment-13466066 ] Hudson commented on HADOOP-8866: Integrated in Hadoop-Hdfs-trunk-Commit #2850 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2850/]) HADOOP-8866. SampleQuantiles#query is O(N^2) instead of O(N). Contributed by Andrew Wang. (Revision 1391711) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1391711 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/SampleQuantiles.java SampleQuantiles#query is O(N^2) instead of O(N) --- Key: HADOOP-8866 URL: https://issues.apache.org/jira/browse/HADOOP-8866 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.2-alpha Reporter: Andrew Wang Assignee: Andrew Wang Priority: Minor Fix For: 2.0.3-alpha Attachments: hdfs-3976-1.patch SampleQuantiles#query() does O(N) calls LinkedList#get() in a loop, rather than using an iterator. This makes query O(N^2), rather than O(N). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8857) hadoop.http.authentication.signature.secret.file should be created if the configured file does not exist
[
https://issues.apache.org/jira/browse/HADOOP-8857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466087#comment-13466087
]
Owen O'Malley commented on HADOOP-8857:
---
Actually, we need to:
* remove the default value from core-default.xml
* if the value isn't defined, let the lower layer generate random bytes
Saving the random bytes lowers the security of the system.
hadoop.http.authentication.signature.secret.file should be created if the
configured file does not exist
Key: HADOOP-8857
URL: https://issues.apache.org/jira/browse/HADOOP-8857
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Priority: Minor
AuthenticationFilterInitializer#initFilter fails if the configured
{{hadoop.http.authentication.signature.secret.file}} does not exist, eg:
{noformat}
java.lang.RuntimeException: Could not read HTTP signature secret file:
/var/lib/hadoop-hdfs/hadoop-http-auth-signature-secret
{noformat}
Creating /var/lib/hadoop-hdfs/hadoop-http-auth-signature-secret (populated
with a string) fixes the issue. Per the auth docs If a secret is not
provided a random secret is generated at start up time., which sounds like
it means the file should be generated at startup with a random secrete, which
doesn't seem to be the case. Also the instructions in the docs should be more
clear in this regard.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-8857) hadoop.http.authentication.signature.secret.file should be created if the configured file does not exist
[
https://issues.apache.org/jira/browse/HADOOP-8857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley reassigned HADOOP-8857:
-
Assignee: Owen O'Malley
hadoop.http.authentication.signature.secret.file should be created if the
configured file does not exist
Key: HADOOP-8857
URL: https://issues.apache.org/jira/browse/HADOOP-8857
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Assignee: Owen O'Malley
Priority: Minor
AuthenticationFilterInitializer#initFilter fails if the configured
{{hadoop.http.authentication.signature.secret.file}} does not exist, eg:
{noformat}
java.lang.RuntimeException: Could not read HTTP signature secret file:
/var/lib/hadoop-hdfs/hadoop-http-auth-signature-secret
{noformat}
Creating /var/lib/hadoop-hdfs/hadoop-http-auth-signature-secret (populated
with a string) fixes the issue. Per the auth docs If a secret is not
provided a random secret is generated at start up time., which sounds like
it means the file should be generated at startup with a random secrete, which
doesn't seem to be the case. Also the instructions in the docs should be more
clear in this regard.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8863) Eclipse plugin may not be working on Juno due to changes in it
[ https://issues.apache.org/jira/browse/HADOOP-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466108#comment-13466108 ] Harsh J commented on HADOOP-8863: - Hey Adam, Indeed, and I supported that move as well. Just needed to track these issues though, until a separate project floats (has a proposal been sent?). Happy to help fix this. The Eclipse I tried it out on, is 3.8. Preferences parts are the ones that are failing, although the plugin loads fine otherwise (rendering it unusable to a certain extent). Eclipse plugin may not be working on Juno due to changes in it -- Key: HADOOP-8863 URL: https://issues.apache.org/jira/browse/HADOOP-8863 Project: Hadoop Common Issue Type: Bug Components: contrib/eclipse-plugin Affects Versions: 1.2.0 Reporter: Harsh J Assignee: Harsh J We need to debug/investigate why it is so. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8865) log warn when loading deprecated properties
[ https://issues.apache.org/jira/browse/HADOOP-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianbin Wei updated HADOOP-8865: Status: Patch Available (was: Open) log warn when loading deprecated properties --- Key: HADOOP-8865 URL: https://issues.apache.org/jira/browse/HADOOP-8865 Project: Hadoop Common Issue Type: Improvement Components: conf Affects Versions: 3.0.0 Reporter: Jianbin Wei Fix For: 3.0.0 Attachments: HADOOP-8865.patch We need to log warn when loading deprecated properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8865) log warn when loading deprecated properties
[ https://issues.apache.org/jira/browse/HADOOP-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianbin Wei updated HADOOP-8865: Attachment: HADOOP-8865.patch log warn when loading deprecated properties --- Key: HADOOP-8865 URL: https://issues.apache.org/jira/browse/HADOOP-8865 Project: Hadoop Common Issue Type: Improvement Components: conf Affects Versions: 3.0.0 Reporter: Jianbin Wei Fix For: 3.0.0 Attachments: HADOOP-8865.patch We need to log warn when loading deprecated properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
