[jira] [Commented] (HADOOP-11090) [Umbrella] Support Java 8 in Hadoop
[ https://issues.apache.org/jira/browse/HADOOP-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496418#comment-14496418 ] Andrew Wang commented on HADOOP-11090: -- We could probably close this JIRA. branch-2 seems to run fine on JDK8, and we've been fixing up compilation issues as we find them. I'm hoping to bump the JDK min version to JDK8 in trunk pretty soon, since as noted earlier JDK7 is EOL this month. [Umbrella] Support Java 8 in Hadoop --- Key: HADOOP-11090 URL: https://issues.apache.org/jira/browse/HADOOP-11090 Project: Hadoop Common Issue Type: New Feature Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Java 8 is coming quickly to various clusters. Making sure Hadoop seamlessly works with Java 8 is important for the Apache community. This JIRA is to track the issues/experiences encountered during Java 8 migration. If you find a potential bug , please create a separate JIRA either as a sub-task or linked into this JIRA. If you find a Hadoop or JVM configuration tuning, you can create a JIRA as well. Or you can add a comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11090) [Umbrella] Support Java 8 in Hadoop
[ https://issues.apache.org/jira/browse/HADOOP-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496530#comment-14496530 ] Tsuyoshi Ozawa commented on HADOOP-11090: - In fact, I think 2.7 is ready for JDK 7 since almost works are done: * HADOOP-11098, HADOOP-11364 are configuration problems. * MAPREDUCE-6165 is only test problem. It's not related to body of Hadoop. Anyway, let's vote about this topic. [Umbrella] Support Java 8 in Hadoop --- Key: HADOOP-11090 URL: https://issues.apache.org/jira/browse/HADOOP-11090 Project: Hadoop Common Issue Type: New Feature Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Java 8 is coming quickly to various clusters. Making sure Hadoop seamlessly works with Java 8 is important for the Apache community. This JIRA is to track the issues/experiences encountered during Java 8 migration. If you find a potential bug , please create a separate JIRA either as a sub-task or linked into this JIRA. If you find a Hadoop or JVM configuration tuning, you can create a JIRA as well. Or you can add a comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-10116) fix inconsistent synchronization warnings in ZlibCompressor
[
https://issues.apache.org/jira/browse/HADOOP-10116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496502#comment-14496502
]
Brahma Reddy Battula commented on HADOOP-10116:
---
[~cmccabe] , can I close this issue..? As part of HADOOP-10681 this got fixed...
fix inconsistent synchronization warnings in ZlibCompressor
-
Key: HADOOP-10116
URL: https://issues.apache.org/jira/browse/HADOOP-10116
Project: Hadoop Common
Issue Type: Bug
Reporter: Colin Patrick McCabe
Fix findbugs warnings in ZlibCompressor. I believe these were introduced by
HADOOP-10047.
{code}
Code Warning
ISInconsistent synchronization of
org.apache.hadoop.io.compress.zlib.ZlibCompressor.keepUncompressedBuf; locked
57% of time
ISInconsistent synchronization of
org.apache.hadoop.io.compress.zlib.ZlibCompressor.userBuf; locked 60% of time
ISInconsistent synchronization of
org.apache.hadoop.io.compress.zlib.ZlibCompressor.userBufLen; locked 85% of
time
ISInconsistent synchronization of
org.apache.hadoop.io.compress.zlib.ZlibDecompressor.userBuf; locked 60% of
time
ISInconsistent synchronization of
org.apache.hadoop.io.compress.zlib.ZlibDecompressor.userBufLen; locked 77% of
time
Dodgy Warnings
Code Warning
DLS Dead store to pos2 in
org.apache.hadoop.io.compress.zlib.ZlibCompressor.put(ByteBuffer, ByteBuffer)
DLS Dead store to pos2 in
org.apache.hadoop.io.compress.zlib.ZlibDecompressor.put(ByteBuffer,
ByteBuffer)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11090) [Umbrella] Support Java 8 in Hadoop
[ https://issues.apache.org/jira/browse/HADOOP-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496495#comment-14496495 ] Arpit Agarwal commented on HADOOP-11090: bq. I'm hoping to bump the JDK min version to JDK8 in trunk pretty soon, since as noted earlier JDK7 is EOL this month. Hi Andrew, it should probably be put to vote (unless the community already agreed to it and I missed it). I assume branch-2 remains on Java 7 for the foreseeable future. Thanks, Arpit. [Umbrella] Support Java 8 in Hadoop --- Key: HADOOP-11090 URL: https://issues.apache.org/jira/browse/HADOOP-11090 Project: Hadoop Common Issue Type: New Feature Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Java 8 is coming quickly to various clusters. Making sure Hadoop seamlessly works with Java 8 is important for the Apache community. This JIRA is to track the issues/experiences encountered during Java 8 migration. If you find a potential bug , please create a separate JIRA either as a sub-task or linked into this JIRA. If you find a Hadoop or JVM configuration tuning, you can create a JIRA as well. Or you can add a comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11090) [Umbrella] Support Java 8 in Hadoop
[ https://issues.apache.org/jira/browse/HADOOP-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496546#comment-14496546 ] Arpit Agarwal commented on HADOOP-11090: bq. In fact, I think 2.7 is ready for JDK 7 since almost works are done: I see, I thought branch-2 cannot be forced to require JDK 8 for compatibility reasons however it might be permitted by our compatibility guide. bq. _The JVM requirements will not change across point releases within the same minor release except if the JVM version under question becomes unsupported. Minor/major releases might require later versions of JVM for some/all of the supported operating systems._ Thanks. [Umbrella] Support Java 8 in Hadoop --- Key: HADOOP-11090 URL: https://issues.apache.org/jira/browse/HADOOP-11090 Project: Hadoop Common Issue Type: New Feature Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Java 8 is coming quickly to various clusters. Making sure Hadoop seamlessly works with Java 8 is important for the Apache community. This JIRA is to track the issues/experiences encountered during Java 8 migration. If you find a potential bug , please create a separate JIRA either as a sub-task or linked into this JIRA. If you find a Hadoop or JVM configuration tuning, you can create a JIRA as well. Or you can add a comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-10013) FileSystem checkPath accepts invalid paths with an authority but no scheme
[
https://issues.apache.org/jira/browse/HADOOP-10013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496516#comment-14496516
]
Brahma Reddy Battula commented on HADOOP-10013:
---
[~daryn] AFAIK this bug not exists in branch-2 and trunk..can we close this
jira..?
{code}
String thatScheme = uri.getScheme();
if (thatScheme == null)// fs is relative
return;
{code}
FileSystem checkPath accepts invalid paths with an authority but no scheme
--
Key: HADOOP-10013
URL: https://issues.apache.org/jira/browse/HADOOP-10013
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Daryn Sharp
{{FileSystem#checkPath}} will consider paths of the form //junk/path as being
valid for the given fs. The problem is {{checkPath}} shorts out if the path
contains no scheme - assuming it must be a relative or absolute path for the
given fs - whereas the condition should be no scheme _and_ no authority.
This causes {{DistributedFileSystem#getPathName}} to convert //junk/path into
/path, which silently hides the use of invalid paths.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11784) failed to locate Winutils for win 32 platform
[ https://issues.apache.org/jira/browse/HADOOP-11784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496907#comment-14496907 ] Kiran Kumar M R commented on HADOOP-11784: -- HADOOP-9922 has added support for WIN32 in Hadoop Common. You can apply this patch and compile hadoop-common to get winutils.exe failed to locate Winutils for win 32 platform - Key: HADOOP-11784 URL: https://issues.apache.org/jira/browse/HADOOP-11784 Project: Hadoop Common Issue Type: Bug Components: bin Affects Versions: 2.6.0 Reporter: Gaurav Tiwari During the execution of Map reduce example first I got the error telling ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path So I downloaded a version and update the bin folder again executing the same command I am getting error like ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable C:\hadoop-2.6.0\bin\winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355) at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370) at org.apache.hadoop.util.Shell.clinit(Shell.java:363) at org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:438) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:484) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:170) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:153) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) at org.apache.hadoop.examples.Grep.main(Grep.java:101) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71) at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 15/04/01 19:12:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where ap licable 15/04/01 19:12:52 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 15/04/01 19:12:52 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= java.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:1011) at org.apache.hadoop.util.Shell.runCommand(Shell.java:482) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at org.apache.hadoop.util.Shell.execCommand(Shell.java:808) at org.apache.hadoop.util.Shell.execCommand(Shell.java:791) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:656) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:444) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:293) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:437) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314) at org.apache.hadoop.examples.Grep.run(Grep.java:77) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at
[jira] [Assigned] (HADOOP-11784) failed to locate Winutils for win 32 platform
[ https://issues.apache.org/jira/browse/HADOOP-11784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kiran Kumar M R reassigned HADOOP-11784: Assignee: Kiran Kumar M R failed to locate Winutils for win 32 platform - Key: HADOOP-11784 URL: https://issues.apache.org/jira/browse/HADOOP-11784 Project: Hadoop Common Issue Type: Bug Components: bin Affects Versions: 2.6.0 Reporter: Gaurav Tiwari Assignee: Kiran Kumar M R During the execution of Map reduce example first I got the error telling ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path So I downloaded a version and update the bin folder again executing the same command I am getting error like ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable C:\hadoop-2.6.0\bin\winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355) at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370) at org.apache.hadoop.util.Shell.clinit(Shell.java:363) at org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:438) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:484) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:170) at org.apache.hadoop.util.GenericOptionsParser.init(GenericOptionsParser.java:153) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) at org.apache.hadoop.examples.Grep.main(Grep.java:101) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71) at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 15/04/01 19:12:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where ap licable 15/04/01 19:12:52 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 15/04/01 19:12:52 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= java.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:1011) at org.apache.hadoop.util.Shell.runCommand(Shell.java:482) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at org.apache.hadoop.util.Shell.execCommand(Shell.java:808) at org.apache.hadoop.util.Shell.execCommand(Shell.java:791) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:656) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:444) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:293) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:437) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314) at org.apache.hadoop.examples.Grep.run(Grep.java:77) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.examples.Grep.main(Grep.java:101) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
[jira] [Commented] (HADOOP-11746) rewrite test-patch.sh
[
https://issues.apache.org/jira/browse/HADOOP-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496745#comment-14496745
]
Chris Nauroth commented on HADOOP-11746:
Thank you for the patch, Allen. The new functionality looks great! Here are a
few comments.
checkstyle.sh
# {{checkstyle_postapply}}: Please correct the indentation on this line.
{code}
cp -p ${BASEDIR}/target/checkstyle-result.xml
${PATCH_DIR}/checkstyle-result-patch.xml
{code}
# Do you think it's worthwhile to move the Python code into its own .py file?
shellcheck.sh
# {{shellcheck_private_findbash}}: This looks for files in bin and sbin. Do we
also need libexec, which is currently used in hadoop-kms and hadoop-hdfs-httpfs?
# There are shellcheck warnings on lines 49 and 70. I see shellcheck.sh caught
them in the last run, so that's cool. :-)
whitespace.sh
# From some quick testing of the grep, it's catching trailing spaces, but not
trailing tabs.
test-patch.sh
# {{precheck_without_patch}} and {{check_site}}: The mvn site command that is
echoed is slightly different from what is actually run. The arguments are in a
different order. I wonder if it would be helpful to have a {{echo_and_exec}}
helper function to call everywhere that we do this kind of thing.
# {{determine_needed_tests}}: I believe this would not identify tests for a
patch that contained only changes in test resources (not Java test code).
Examples of this include testConf.xml and editsStored and editsStored.xml.
# {{check_unittests}}: The echo of the mvn command on line 1736 does not
include the redirection of output to test_logfile as the actual executed
command does.
# {{output_to_console}}: Today I learned that your secret talent is ASCII
elephant art. :-)
# There are shellcheck warnings on lines 1158, 1573, 1732, 1751, and 1995.
# The old test-patch.sh output always included a hyperlink to the patch file
that it ran. Can we please add that? I always found that helpful for knowing
exactly which patch it was reporting on, in case multiple revisions got
uploaded quickly.
Are you planning to clean up smart-apply-patch.sh in scope of this jira, or
will that happen elsewhere?
rewrite test-patch.sh
-
Key: HADOOP-11746
URL: https://issues.apache.org/jira/browse/HADOOP-11746
Project: Hadoop Common
Issue Type: Test
Components: build, test
Affects Versions: 3.0.0
Reporter: Allen Wittenauer
Assignee: Allen Wittenauer
Attachments: HADOOP-11746-00.patch, HADOOP-11746-01.patch,
HADOOP-11746-02.patch, HADOOP-11746-03.patch, HADOOP-11746-04.patch,
HADOOP-11746-05.patch, HADOOP-11746-06.patch, HADOOP-11746-07.patch,
HADOOP-11746-09.patch, HADOOP-11746-10.patch, HADOOP-11746-11.patch,
HADOOP-11746-12.patch, HADOOP-11746-13.patch, HADOOP-11746-14.patch,
HADOOP-11746-15.patch
This code is bad and you should feel bad.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11835) hadoop version command outputs Subversion Unknown -r Unknown
Akira AJISAKA created HADOOP-11835:
--
Summary: hadoop version command outputs Subversion Unknown -r
Unknown
Key: HADOOP-11835
URL: https://issues.apache.org/jira/browse/HADOOP-11835
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Akira AJISAKA
Built Hadoop 2.7.0-RC0 from source and executed hadoop version.
{code}
[root@branch-2 ~]# hadoop version
Hadoop 2.7.0
Subversion Unknown -r Unknown
Compiled by aajisaka on 2015-04-14T08:49Z
Compiled with protoc 2.5.0
From source with checksum a9e90912c37a35c3195d23951fd18f
This command was run using
/usr/local/hadoop-2.7.0-rc0/share/hadoop/common/hadoop-common-2.7.0.jar
{code}
Subversion is no longer used for Hadoop, so the line should be removed or
replaced by Git.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11837) After HADOOP-11754, oozie fails to stop cleanly
[ https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Zhang updated HADOOP-11837: - Status: Patch Available (was: Open) After HADOOP-11754, oozie fails to stop cleanly --- Key: HADOOP-11837 URL: https://issues.apache.org/jira/browse/HADOOP-11837 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 2.7.0 Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Priority: Blocker Fix For: 2.7.0 Attachments: HADOOP-11837.patch After HADOOP-11754, AuthenticationFilter has to be enhanced to destroy to secret provider. Else, products like Oozie which extend the AuthenticationFilter fail to stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11835) hadoop version command outputs Subversion Unknown -r Unknown
[
https://issues.apache.org/jira/browse/HADOOP-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated HADOOP-11835:
---
Target Version/s: 2.7.1
hadoop version command outputs Subversion Unknown -r Unknown
--
Key: HADOOP-11835
URL: https://issues.apache.org/jira/browse/HADOOP-11835
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Akira AJISAKA
Assignee: Kiran Kumar M R
Built Hadoop 2.7.0-RC0 from source and executed hadoop version.
{code}
[root@branch-2 ~]# hadoop version
Hadoop 2.7.0
Subversion Unknown -r Unknown
Compiled by aajisaka on 2015-04-14T08:49Z
Compiled with protoc 2.5.0
From source with checksum a9e90912c37a35c3195d23951fd18f
This command was run using
/usr/local/hadoop-2.7.0-rc0/share/hadoop/common/hadoop-common-2.7.0.jar
{code}
Subversion is no longer used for Hadoop, so the line should be removed or
replaced by Git.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11836) Update index.md.vm
[ https://issues.apache.org/jira/browse/HADOOP-11836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated HADOOP-11836: --- Target Version/s: 2.7.1 Update index.md.vm -- Key: HADOOP-11836 URL: https://issues.apache.org/jira/browse/HADOOP-11836 Project: Hadoop Common Issue Type: Bug Components: documentation Affects Versions: 2.6.0, 2.7.0 Reporter: Akira AJISAKA http://hadoop.apache.org/docs/current/ still shows the new features of Hadoop 2.5. The document should be updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11772) RPC Invoker relies on static ClientCache which has synchronized(this) blocks
[
https://issues.apache.org/jira/browse/HADOOP-11772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao updated HADOOP-11772:
---
Target Version/s: 2.7.1 (was: 2.8.0)
RPC Invoker relies on static ClientCache which has synchronized(this) blocks
Key: HADOOP-11772
URL: https://issues.apache.org/jira/browse/HADOOP-11772
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc, performance
Reporter: Gopal V
Assignee: Akira AJISAKA
Attachments: HADOOP-11772-001.patch, HADOOP-11772-wip-001.patch,
HADOOP-11772-wip-002.patch, after-ipc-fix.png, dfs-sync-ipc.png,
sync-client-bt.png, sync-client-threads.png
{code}
private static ClientCache CLIENTS=new ClientCache();
...
this.client = CLIENTS.getClient(conf, factory);
{code}
Meanwhile in ClientCache
{code}
public synchronized Client getClient(Configuration conf,
SocketFactory factory, Class? extends Writable valueClass) {
...
Client client = clients.get(factory);
if (client == null) {
client = new Client(valueClass, conf, factory);
clients.put(factory, client);
} else {
client.incCount();
}
{code}
All invokers end up calling these methods, resulting in IPC clients choking
up.
!sync-client-threads.png!
!sync-client-bt.png!
!dfs-sync-ipc.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11835) hadoop version command outputs Subversion Unknown -r Unknown
[
https://issues.apache.org/jira/browse/HADOOP-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kiran Kumar M R updated HADOOP-11835:
-
Attachment: HADOOP-11835-branch2-001.patch
This patch is for branch-2, branch-2.7.0 and branch-2.7
Trunk already has solution for this issue in JIRA HADOOP-11041
Solution is same
System.out.println(Subversion + getUrl() + -r + getRevision());
is changed to:
System.out.println(Source code repository: + getUrl() + -r +
getRevision());
hadoop version command outputs Subversion Unknown -r Unknown
--
Key: HADOOP-11835
URL: https://issues.apache.org/jira/browse/HADOOP-11835
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Akira AJISAKA
Assignee: Kiran Kumar M R
Attachments: HADOOP-11835-branch2-001.patch
Built Hadoop 2.7.0-RC0 from source and executed hadoop version.
{code}
[root@branch-2 ~]# hadoop version
Hadoop 2.7.0
Subversion Unknown -r Unknown
Compiled by aajisaka on 2015-04-14T08:49Z
Compiled with protoc 2.5.0
From source with checksum a9e90912c37a35c3195d23951fd18f
This command was run using
/usr/local/hadoop-2.7.0-rc0/share/hadoop/common/hadoop-common-2.7.0.jar
{code}
Subversion is no longer used for Hadoop, so the line should be removed or
replaced by Git.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11835) hadoop version command outputs Subversion Unknown -r Unknown
[
https://issues.apache.org/jira/browse/HADOOP-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496857#comment-14496857
]
Akira AJISAKA commented on HADOOP-11835:
Thanks [~kiranmr] for pointing HADOOP-11041. Looking into the issue, it's not
safe to change the output format of the hadoop command. Therefore I'd like to
close this issue as invalid. Sorry for confusion.
hadoop version command outputs Subversion Unknown -r Unknown
--
Key: HADOOP-11835
URL: https://issues.apache.org/jira/browse/HADOOP-11835
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Akira AJISAKA
Assignee: Kiran Kumar M R
Attachments: HADOOP-11835-branch2-001.patch
Built Hadoop 2.7.0-RC0 from source and executed hadoop version.
{code}
[root@branch-2 ~]# hadoop version
Hadoop 2.7.0
Subversion Unknown -r Unknown
Compiled by aajisaka on 2015-04-14T08:49Z
Compiled with protoc 2.5.0
From source with checksum a9e90912c37a35c3195d23951fd18f
This command was run using
/usr/local/hadoop-2.7.0-rc0/share/hadoop/common/hadoop-common-2.7.0.jar
{code}
Subversion is no longer used for Hadoop, so the line should be removed or
replaced by Git.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HADOOP-11835) hadoop version command outputs Subversion Unknown -r Unknown
[
https://issues.apache.org/jira/browse/HADOOP-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA resolved HADOOP-11835.
Resolution: Invalid
hadoop version command outputs Subversion Unknown -r Unknown
--
Key: HADOOP-11835
URL: https://issues.apache.org/jira/browse/HADOOP-11835
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Akira AJISAKA
Assignee: Kiran Kumar M R
Attachments: HADOOP-11835-branch2-001.patch
Built Hadoop 2.7.0-RC0 from source and executed hadoop version.
{code}
[root@branch-2 ~]# hadoop version
Hadoop 2.7.0
Subversion Unknown -r Unknown
Compiled by aajisaka on 2015-04-14T08:49Z
Compiled with protoc 2.5.0
From source with checksum a9e90912c37a35c3195d23951fd18f
This command was run using
/usr/local/hadoop-2.7.0-rc0/share/hadoop/common/hadoop-common-2.7.0.jar
{code}
Subversion is no longer used for Hadoop, so the line should be removed or
replaced by Git.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HADOOP-11835) hadoop version command outputs Subversion Unknown -r Unknown
[
https://issues.apache.org/jira/browse/HADOOP-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kiran Kumar M R reassigned HADOOP-11835:
Assignee: Kiran Kumar M R
hadoop version command outputs Subversion Unknown -r Unknown
--
Key: HADOOP-11835
URL: https://issues.apache.org/jira/browse/HADOOP-11835
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Akira AJISAKA
Assignee: Kiran Kumar M R
Built Hadoop 2.7.0-RC0 from source and executed hadoop version.
{code}
[root@branch-2 ~]# hadoop version
Hadoop 2.7.0
Subversion Unknown -r Unknown
Compiled by aajisaka on 2015-04-14T08:49Z
Compiled with protoc 2.5.0
From source with checksum a9e90912c37a35c3195d23951fd18f
This command was run using
/usr/local/hadoop-2.7.0-rc0/share/hadoop/common/hadoop-common-2.7.0.jar
{code}
Subversion is no longer used for Hadoop, so the line should be removed or
replaced by Git.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11836) Update index.md.vm for Hadoop 2.7
Akira AJISAKA created HADOOP-11836: -- Summary: Update index.md.vm for Hadoop 2.7 Key: HADOOP-11836 URL: https://issues.apache.org/jira/browse/HADOOP-11836 Project: Hadoop Common Issue Type: Bug Components: documentation Affects Versions: 2.7.0 Reporter: Akira AJISAKA http://hadoop.apache.org/docs/current/ still shows the new features of Hadoop 2.4. The document should be updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11836) Update index.md.vm
[ https://issues.apache.org/jira/browse/HADOOP-11836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated HADOOP-11836: --- Description: http://hadoop.apache.org/docs/current/ still shows the new features of Hadoop 2.5. The document should be updated. (was: http://hadoop.apache.org/docs/current/ still shows the new features of Hadoop 2.4. The document should be updated.) Affects Version/s: 2.6.0 Summary: Update index.md.vm (was: Update index.md.vm for Hadoop 2.7) Update index.md.vm -- Key: HADOOP-11836 URL: https://issues.apache.org/jira/browse/HADOOP-11836 Project: Hadoop Common Issue Type: Bug Components: documentation Affects Versions: 2.6.0, 2.7.0 Reporter: Akira AJISAKA http://hadoop.apache.org/docs/current/ still shows the new features of Hadoop 2.5. The document should be updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-11837) After HADOOP-11754, oozie fails to stop cleanly
Venkat Ranganathan created HADOOP-11837: --- Summary: After HADOOP-11754, oozie fails to stop cleanly Key: HADOOP-11837 URL: https://issues.apache.org/jira/browse/HADOOP-11837 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 2.7.0 Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Priority: Blocker Fix For: 2.7.0 After HADOOP-11754, AuthenticationFilter has to be enhanced to destroy to secret provider. Else, products like Oozie which extend the AuthenticationFilter fail to stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11837) After HADOOP-11754, oozie fails to stop cleanly
[ https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkat Ranganathan updated HADOOP-11837: Attachment: HADOOP-11837.patch After HADOOP-11754, oozie fails to stop cleanly --- Key: HADOOP-11837 URL: https://issues.apache.org/jira/browse/HADOOP-11837 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 2.7.0 Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Priority: Blocker Fix For: 2.7.0 Attachments: HADOOP-11837.patch After HADOOP-11754, AuthenticationFilter has to be enhanced to destroy to secret provider. Else, products like Oozie which extend the AuthenticationFilter fail to stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Attachment: HADOOP-11120 Suggests that the correct replacement for the deprecates `-rmr` command is `-rm -r`, not `rm -r` hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Dasha Boudnik Attachments: HADOOP-11120, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland reassigned HADOOP-11120: Assignee: Juliet Hougland (was: Dasha Boudnik) hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: HADOOP-11120, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Status: Open (was: Patch Available) hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Status: Patch Available (was: Open) hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Attachment: HADOOP-11120.patch hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Attachment: (was: HADOOP-11120) hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Status: Patch Available (was: Open) This is a change to the deprecation message for `hadoop fs -rmr` that corrects the suggested replacement command to be `hadoop fs -rm -r` hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[
https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497291#comment-14497291
]
Hadoop QA commented on HADOOP-11120:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725729/HADOOP-11120.patch
against trunk revision 1b89a3e.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6106//console
This message is automatically generated.
hadoop fs -rmr gives wrong advice
-
Key: HADOOP-11120
URL: https://issues.apache.org/jira/browse/HADOOP-11120
Project: Hadoop Common
Issue Type: Bug
Reporter: Allen Wittenauer
Assignee: Juliet Hougland
Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21
PM.png
Typing bin/hadoop fs -rmr /a?
gives the output:
rmr: DEPRECATED: Please use 'rm -r' instead.
Typing bin/hadoop fs rm -r /a?
gives the output:
rm: Unknown command
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497293#comment-14497293 ] Juliet Hougland commented on HADOOP-11120: -- Hmm, looks like I need to regenerate the diff. hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Attachment: (was: HADOOP-11120.patch) hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-11838) Externalize AuthToken interface
Kai Zheng created HADOOP-11838:
--
Summary: Externalize AuthToken interface
Key: HADOOP-11838
URL: https://issues.apache.org/jira/browse/HADOOP-11838
Project: Hadoop Common
Issue Type: Sub-task
Reporter: Kai Zheng
Assignee: Kai Zheng
This suggests refactoring existing {{AuthToken}} utility class and
externalizing a generic {{AuthToken}} API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11838) Externalize AuthToken interface
[
https://issues.apache.org/jira/browse/HADOOP-11838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kai Zheng updated HADOOP-11838:
---
Attachment: HADOOP-11838-v1.patch
Uploaded an initial patch to illustrate the idea:
1. Have a new package for all the new token stuffs, in it adding a
{{AuthToken}} interface for now;
2. Renamed the existing {{AuthToken}} utility class as {{DefaultAuthToken}},
implementing the new interface. This class currently is only used internally,
so it's safe.
Externalize AuthToken interface
---
Key: HADOOP-11838
URL: https://issues.apache.org/jira/browse/HADOOP-11838
Project: Hadoop Common
Issue Type: Sub-task
Components: security
Reporter: Kai Zheng
Assignee: Kai Zheng
Attachments: HADOOP-11838-v1.patch
This suggests refactoring existing {{AuthToken}} utility class and
externalizing a generic {{AuthToken}} API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Attachment: HADOOP-11120.patch This should be reviewed, please. hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[
https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497379#comment-14497379
]
Hadoop QA commented on HADOOP-11120:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725735/HADOOP-11120.patch
against trunk revision 1b89a3e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.cli.TestCLI
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6107//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6107//console
This message is automatically generated.
hadoop fs -rmr gives wrong advice
-
Key: HADOOP-11120
URL: https://issues.apache.org/jira/browse/HADOOP-11120
Project: Hadoop Common
Issue Type: Bug
Reporter: Allen Wittenauer
Assignee: Juliet Hougland
Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21
PM.png
Typing bin/hadoop fs -rmr /a?
gives the output:
rmr: DEPRECATED: Please use 'rm -r' instead.
Typing bin/hadoop fs rm -r /a?
gives the output:
rm: Unknown command
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Status: Open (was: Patch Available) hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Attachment: HADOOP-11120.patch hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Status: Patch Available (was: Open) hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[ https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland updated HADOOP-11120: - Attachment: (was: HADOOP-11120.patch) hadoop fs -rmr gives wrong advice - Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue Type: Bug Reporter: Allen Wittenauer Assignee: Juliet Hougland Attachments: Screen Shot 2014-09-24 at 3.02.21 PM.png Typing bin/hadoop fs -rmr /a? gives the output: rmr: DEPRECATED: Please use 'rm -r' instead. Typing bin/hadoop fs rm -r /a? gives the output: rm: Unknown command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Moved] (HADOOP-11839) when try to roll one key which not exist in kms ,will have nullpointer Exception
[ https://issues.apache.org/jira/browse/HADOOP-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nijel moved HDFS-8158 to HADOOP-11839: -- Component/s: (was: encryption) kms Affects Version/s: (was: 2.6.0) 2.6.0 Key: HADOOP-11839 (was: HDFS-8158) Project: Hadoop Common (was: Hadoop HDFS) when try to roll one key which not exist in kms ,will have nullpointer Exception Key: HADOOP-11839 URL: https://issues.apache.org/jira/browse/HADOOP-11839 Project: Hadoop Common Issue Type: Bug Components: kms Affects Versions: 2.6.0 Reporter: huangyitian Assignee: J.Andreina Priority: Minor Test Step: 1.try to roll one key which is not existed in kms: ./hadoop key roll hyt Test reslt: will have a nullPointer Exception in Linux consol: vm-204:/opt/OpenSource/install/hadoop/namenode/bin # ./hadoop key roll hyt 15/04/16 11:58:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Rolling key version from KeyProvider: KMSClientProvider[http://9.91.8.204:16000/kms/v1/] for key name: hyt java.lang.NullPointerException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:485) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:443) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.rollNewVersionInternal(KMSClientProvider.java:649) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.rollNewVersion(KMSClientProvider.java:660) at org.apache.hadoop.crypto.key.KeyShell$RollCommand.execute(KeyShell.java:347) at org.apache.hadoop.crypto.key.KeyShell.run(KeyShell.java:79) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.crypto.key.KeyShell.main(KeyShell.java:515) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11834) Add ErasureCodecFactory to create ErasureCodec using codec's short name.
[
https://issues.apache.org/jira/browse/HADOOP-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497374#comment-14497374
]
Kai Zheng commented on HADOOP-11834:
Hi [~umamaheswararao] and [~vinayrpet],
I'm not very sure about this. In HADOOP-11649 I prototyped a factory-like class
{{ErasureCodecLoader}} which contains a method {{getCodecByName}}, which could
serve for this purpose. I would update the patch there recently for your review
and further discussion. Thanks.
Add ErasureCodecFactory to create ErasureCodec using codec's short name.
Key: HADOOP-11834
URL: https://issues.apache.org/jira/browse/HADOOP-11834
Project: Hadoop Common
Issue Type: Sub-task
Components: io
Reporter: Vinayakumar B
Assignee: Vinayakumar B
Using Codec's shortname Codec instance should be create.
such as, using name rs {{RSErasureCodec}} should be created.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11837) After HADOOP-11754, oozie fails to stop cleanly
[ https://issues.apache.org/jira/browse/HADOOP-11837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497454#comment-14497454 ] Venkat Ranganathan commented on HADOOP-11837: - Thanks [~wheat9] for the quick review. Not sure I understand your comment. HttpServer2 creates a new instance of the signer provider using constructSecretProvider and will destroy that object. AuthenticationFilter instance is not exposed to others. Why would you need a flag to protect the private instance in the destroy method. After HADOOP-11754, oozie fails to stop cleanly --- Key: HADOOP-11837 URL: https://issues.apache.org/jira/browse/HADOOP-11837 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 2.7.0 Reporter: Venkat Ranganathan Assignee: Venkat Ranganathan Priority: Blocker Fix For: 2.7.0 Attachments: HADOOP-11837.patch After HADOOP-11754, AuthenticationFilter has to be enhanced to destroy to secret provider. Else, products like Oozie which extend the AuthenticationFilter fail to stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11120) hadoop fs -rmr gives wrong advice
[
https://issues.apache.org/jira/browse/HADOOP-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497521#comment-14497521
]
Hadoop QA commented on HADOOP-11120:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725769/HADOOP-11120.patch
against trunk revision 1b89a3e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6109//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6109//console
This message is automatically generated.
hadoop fs -rmr gives wrong advice
-
Key: HADOOP-11120
URL: https://issues.apache.org/jira/browse/HADOOP-11120
Project: Hadoop Common
Issue Type: Bug
Reporter: Allen Wittenauer
Assignee: Juliet Hougland
Attachments: HADOOP-11120.patch, Screen Shot 2014-09-24 at 3.02.21
PM.png
Typing bin/hadoop fs -rmr /a?
gives the output:
rmr: DEPRECATED: Please use 'rm -r' instead.
Typing bin/hadoop fs rm -r /a?
gives the output:
rm: Unknown command
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-10597) Evaluate if we can have RPC client back off when server is under heavy load
[
https://issues.apache.org/jira/browse/HADOOP-10597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14495770#comment-14495770
]
Hadoop QA commented on HADOOP-10597:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725491/HADOOP-10597-6.patch
against trunk revision fddd552.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6104//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6104//console
This message is automatically generated.
Evaluate if we can have RPC client back off when server is under heavy load
---
Key: HADOOP-10597
URL: https://issues.apache.org/jira/browse/HADOOP-10597
Project: Hadoop Common
Issue Type: Sub-task
Reporter: Ming Ma
Assignee: Ming Ma
Attachments: HADOOP-10597-2.patch, HADOOP-10597-3.patch,
HADOOP-10597-4.patch, HADOOP-10597-5.patch, HADOOP-10597-6.patch,
HADOOP-10597.patch, MoreRPCClientBackoffEvaluation.pdf,
RPCClientBackoffDesignAndEvaluation.pdf
Currently if an application hits NN too hard, RPC requests be in blocking
state, assuming OS connection doesn't run out. Alternatively RPC or NN can
throw some well defined exception back to the client based on certain
policies when it is under heavy load; client will understand such exception
and do exponential back off, as another implementation of
RetryInvocationHandler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-10597) Evaluate if we can have RPC client back off when server is under heavy load
[ https://issues.apache.org/jira/browse/HADOOP-10597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HADOOP-10597: - Attachment: HADOOP-10597-6.patch Here is the updated patch with Steve's suggestion. Thanks, Steve and Arpit. Evaluate if we can have RPC client back off when server is under heavy load --- Key: HADOOP-10597 URL: https://issues.apache.org/jira/browse/HADOOP-10597 Project: Hadoop Common Issue Type: Sub-task Reporter: Ming Ma Assignee: Ming Ma Attachments: HADOOP-10597-2.patch, HADOOP-10597-3.patch, HADOOP-10597-4.patch, HADOOP-10597-5.patch, HADOOP-10597-6.patch, HADOOP-10597.patch, MoreRPCClientBackoffEvaluation.pdf, RPCClientBackoffDesignAndEvaluation.pdf Currently if an application hits NN too hard, RPC requests be in blocking state, assuming OS connection doesn't run out. Alternatively RPC or NN can throw some well defined exception back to the client based on certain policies when it is under heavy load; client will understand such exception and do exponential back off, as another implementation of RetryInvocationHandler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11698) remove distcpv1 from hadoop-extras
[ https://issues.apache.org/jira/browse/HADOOP-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14495774#comment-14495774 ] Todd Lipcon commented on HADOOP-11698: -- [~aw] - I'm still around, though :) That said, I don't know what Logalyzer is... its obsolescence predates me. So, I have no opinion on this matter :) remove distcpv1 from hadoop-extras -- Key: HADOOP-11698 URL: https://issues.apache.org/jira/browse/HADOOP-11698 Project: Hadoop Common Issue Type: Improvement Components: tools/distcp Affects Versions: 3.0.0 Reporter: Allen Wittenauer Assignee: Brahma Reddy Battula Attachments: HADOOP-11698-branch2.patch, HADOOP-11698.patch distcpv1 is pretty much unsupported. we should just remove it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11802) DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm
[
https://issues.apache.org/jira/browse/HADOOP-11802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14495835#comment-14495835
]
Hadoop QA commented on HADOOP-11802:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725472/HADOOP-11802.002.patch
against trunk revision fddd552.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6103//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6103//console
This message is automatically generated.
DomainSocketWatcher thread terminates sometimes after there is an I/O error
during requestShortCircuitShm
-
Key: HADOOP-11802
URL: https://issues.apache.org/jira/browse/HADOOP-11802
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Eric Payne
Assignee: Colin Patrick McCabe
Attachments: HADOOP-11802.001.patch, HADOOP-11802.002.patch
In {{DataXceiver#requestShortCircuitShm}}, we attempt to recover from some
errors by closing the {{DomainSocket}}. However, this violates the invariant
that the domain socket should never be closed when it is being managed by the
{{DomainSocketWatcher}}. Instead, we should call {{shutdown}} on the
{{DomainSocket}}. When this bug hits, it terminates the
{{DomainSocketWatcher}} thread.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HADOOP-11834) Add ErasureCodecFactory to create ErasureCodec using codec's short name.
[
https://issues.apache.org/jira/browse/HADOOP-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B reassigned HADOOP-11834:
--
Assignee: Vinayakumar B
Add ErasureCodecFactory to create ErasureCodec using codec's short name.
Key: HADOOP-11834
URL: https://issues.apache.org/jira/browse/HADOOP-11834
Project: Hadoop Common
Issue Type: Sub-task
Components: io
Reporter: Vinayakumar B
Assignee: Vinayakumar B
Using Codec's shortname Codec instance should be create.
such as, using name rs {{RSErasureCodec}} should be created.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11834) Add ErasureCodecFactory to create ErasureCodec using codec's short name.
Vinayakumar B created HADOOP-11834:
--
Summary: Add ErasureCodecFactory to create ErasureCodec using
codec's short name.
Key: HADOOP-11834
URL: https://issues.apache.org/jira/browse/HADOOP-11834
Project: Hadoop Common
Issue Type: Sub-task
Reporter: Vinayakumar B
Using Codec's shortname Codec instance should be create.
such as, using name rs {{RSErasureCodec}} should be created.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
