[jira] [Updated] (HADOOP-8568) DNS#reverseDns fails on IPv6 addresses
[
https://issues.apache.org/jira/browse/HADOOP-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Collins updated HADOOP-8568:
Assignee: (was: Eli Collins)
Labels: newbie (was: )
DNS#reverseDns fails on IPv6 addresses
--
Key: HADOOP-8568
URL: https://issues.apache.org/jira/browse/HADOOP-8568
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Labels: newbie
DNS#reverseDns assumes hostIp is a v4 address (4 parts separated by dots),
blows up if given a v6 address:
{noformat}
Caused by: java.lang.ArrayIndexOutOfBoundsException: 3
at org.apache.hadoop.net.DNS.reverseDns(DNS.java:79)
at org.apache.hadoop.net.DNS.getHosts(DNS.java:237)
at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:340)
at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:358)
at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:337)
at org.apache.hadoop.hbase.master.HMaster.init(HMaster.java:235)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1649)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8567) Backport conf servlet with dump running configuration to branch 1.x
[ https://issues.apache.org/jira/browse/HADOOP-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-8567: Target Version/s: 1.2.0 Fix Version/s: (was: 2.1.0-alpha) (was: 0.21.1) Backport conf servlet with dump running configuration to branch 1.x --- Key: HADOOP-8567 URL: https://issues.apache.org/jira/browse/HADOOP-8567 Project: Hadoop Common Issue Type: New Feature Components: conf Affects Versions: 1.0.0 Reporter: Junping Du Assignee: Junping Du HADOOP-6408 provide conf servlet that can dump running configuration which great helps admin to trouble shooting the configuration issue. However, that patch works on branch after 0.21 only and should be backport to branch 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8567) Backport conf servlet with dump running configuration to branch 1.x
[ https://issues.apache.org/jira/browse/HADOOP-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J updated HADOOP-8567: Affects Version/s: (was: 1.0.3) 1.0.0 Backport conf servlet with dump running configuration to branch 1.x --- Key: HADOOP-8567 URL: https://issues.apache.org/jira/browse/HADOOP-8567 Project: Hadoop Common Issue Type: New Feature Components: conf Affects Versions: 1.0.0 Reporter: Junping Du Assignee: Junping Du HADOOP-6408 provide conf servlet that can dump running configuration which great helps admin to trouble shooting the configuration issue. However, that patch works on branch after 0.21 only and should be backport to branch 1.x. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8551) fs -mkdir creates parent directories without the -p option
[ https://issues.apache.org/jira/browse/HADOOP-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419094#comment-13419094 ] Hudson commented on HADOOP-8551: Integrated in Hadoop-Hdfs-0.23-Build #319 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/319/]) Reverting HADOOP-8551. fs -mkdir creates parent directories without the -p option (Revision 1363550) svn merge -c 1363435 FIXES: HADOOP-8551. fs -mkdir creates parent directories without the -p option (Daryn Sharp and John George via bobby) (Revision 1363438) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1363550 Files : * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Mkdir.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1363438 Files : * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Mkdir.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml fs -mkdir creates parent directories without the -p option -- Key: HADOOP-8551 URL: https://issues.apache.org/jira/browse/HADOOP-8551 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 0.23.3, 2.1.0-alpha, 3.0.0 Reporter: Robert Joseph Evans Assignee: Daryn Sharp Fix For: 0.23.3, 3.0.0, 2.2.0-alpha Attachments: HADOOP-8551.patch, HADOOP-8551.patch hadoop fs -mkdir foo/bar will work even if bar is not present. It should only work if -p is given and foo is not present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8611) Allow fall-back to the shell-based implementation when JNI-based users-group mapping fails
Kihwal Lee created HADOOP-8611: -- Summary: Allow fall-back to the shell-based implementation when JNI-based users-group mapping fails Key: HADOOP-8611 URL: https://issues.apache.org/jira/browse/HADOOP-8611 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 2.0.0-alpha, 0.23.0, 1.0.3 Reporter: Kihwal Lee Assignee: Kihwal Lee Fix For: 1.1.1, 0.23.3, 3.0.0, 2.2.0-alpha When the JNI-based users-group mapping is enabled, the process/command will fail if the native library, libhadoop.so, cannot be found. This mostly happens at client-side where users may use hadoop programatically. Instead of failing, falling back to the shell-based implementation will be desirable. Depending on how cluster is configured, use of the native netgroup mapping cannot be subsituted by the shell-based default. For this reason, this behavior must be configurable with the default being disabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8551) fs -mkdir creates parent directories without the -p option
[ https://issues.apache.org/jira/browse/HADOOP-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419161#comment-13419161 ] Hudson commented on HADOOP-8551: Integrated in Hadoop-Mapreduce-trunk #1142 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1142/]) Reverting HADOOP-8551. fs -mkdir creates parent directories without the -p option (Revision 1363547) HADOOP-8551. fs -mkdir creates parent directories without the -p option (Daryn Sharp and John George via bobby) (Revision 1363435) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1363547 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Mkdir.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1363435 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Mkdir.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml fs -mkdir creates parent directories without the -p option -- Key: HADOOP-8551 URL: https://issues.apache.org/jira/browse/HADOOP-8551 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 0.23.3, 2.1.0-alpha, 3.0.0 Reporter: Robert Joseph Evans Assignee: Daryn Sharp Fix For: 0.23.3, 3.0.0, 2.2.0-alpha Attachments: HADOOP-8551.patch, HADOOP-8551.patch hadoop fs -mkdir foo/bar will work even if bar is not present. It should only work if -p is given and foo is not present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8604) conf/* files overwritten at Hadoop compilation
[ https://issues.apache.org/jira/browse/HADOOP-8604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419187#comment-13419187 ] Jason Lowe commented on HADOOP-8604: The conf/ directory within the Hadoop tree is intended to hold the sample/default configuration. During development, custom configuration changes are not intended to be placed in that directory, as those changes will be overwritten during builds. Usually custom configuration is placed in a separate directory, outside the Hadoop tree, and conveyed to Hadoop via the HADOOP_CONF_DIR environment variable. You can also specify a config directory via the --config option to the hadoop front-end script. With a separate config directory, your changes won't be overwritten during compilation. conf/* files overwritten at Hadoop compilation -- Key: HADOOP-8604 URL: https://issues.apache.org/jira/browse/HADOOP-8604 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 1.0.3 Reporter: Robert Grandl Priority: Minor Whenever I compile hadoop from terminal as: ant compile jar run all the conf/* files are overwritten. I am not sure if some of them should not be like that but at least hadoop-env.sh, mapred-site.ml, core-site.xml, hdfs-site.xml, masters, slaves should remains. Otherwise I am forced to backup and replace content again after compilation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8397) NPE thrown when IPC layer gets an EOF reading a response
[
https://issues.apache.org/jira/browse/HADOOP-8397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HADOOP-8397:
Resolution: Duplicate
Status: Resolved (was: Patch Available)
Sorry, I forgot about this one and it looks like the bug got fixed as part of
HDFS-3504.
NPE thrown when IPC layer gets an EOF reading a response
Key: HADOOP-8397
URL: https://issues.apache.org/jira/browse/HADOOP-8397
Project: Hadoop Common
Issue Type: Bug
Components: ipc
Affects Versions: 2.0.0-alpha, 3.0.0
Reporter: Todd Lipcon
Assignee: Vinay
Priority: Critical
Attachments: HADOOP-8397.patch
When making a call on an IPC connection where the other end has shut down, I
see the following exception:
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:852)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:781)
from the lines:
{code}
RpcResponseHeaderProto response =
RpcResponseHeaderProto.parseDelimitedFrom(in);
int callId = response.getCallId();
{code}
This is because parseDelimitedFrom() returns null in the case that the next
thing to be read on the stream is an EOF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8609) IPC server logs a useless message when shutting down socket
[
https://issues.apache.org/jira/browse/HADOOP-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HADOOP-8609:
---
Resolution: Fixed
Fix Version/s: 2.2.0-alpha
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
The test failure is unrelated. No tests needed since this is just a simple
logging change.
+1, I've just committed committed this to trunk and branch-2.
IPC server logs a useless message when shutting down socket
---
Key: HADOOP-8609
URL: https://issues.apache.org/jira/browse/HADOOP-8609
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 2.0.0-alpha
Reporter: Todd Lipcon
Assignee: Jon Zuanich
Labels: newbie
Fix For: 2.2.0-alpha
Attachments: hadoop-8609.txt
I occasionally see this WARN message out of the NameNode:
{code}
12/07/19 10:37:37 WARN ipc.Server: Ignoring socket shutdown exception
{code}
with no further details. This message isn't useful - it either needs to have
more contextual information (eg what client it's talking about and the
exception ignored), or should be reduced to debug level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8609) IPC server logs a useless message when shutting down socket
[
https://issues.apache.org/jira/browse/HADOOP-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HADOOP-8609:
---
Target Version/s: 2.2.0-alpha
Affects Version/s: (was: 3.0.0)
2.0.0-alpha
IPC server logs a useless message when shutting down socket
---
Key: HADOOP-8609
URL: https://issues.apache.org/jira/browse/HADOOP-8609
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 2.0.0-alpha
Reporter: Todd Lipcon
Assignee: Jon Zuanich
Labels: newbie
Fix For: 2.2.0-alpha
Attachments: hadoop-8609.txt
I occasionally see this WARN message out of the NameNode:
{code}
12/07/19 10:37:37 WARN ipc.Server: Ignoring socket shutdown exception
{code}
with no further details. This message isn't useful - it either needs to have
more contextual information (eg what client it's talking about and the
exception ignored), or should be reduced to debug level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8609) IPC server logs a useless message when shutting down socket
[
https://issues.apache.org/jira/browse/HADOOP-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419559#comment-13419559
]
Hudson commented on HADOOP-8609:
Integrated in Hadoop-Common-trunk-Commit #2509 (See
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2509/])
HADOOP-8609. IPC server logs a useless message when shutting down socket.
Contributed by Jon Zuanich. (Revision 1363950)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1363950
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
IPC server logs a useless message when shutting down socket
---
Key: HADOOP-8609
URL: https://issues.apache.org/jira/browse/HADOOP-8609
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 2.0.0-alpha
Reporter: Todd Lipcon
Assignee: Jon Zuanich
Labels: newbie
Fix For: 2.2.0-alpha
Attachments: hadoop-8609.txt
I occasionally see this WARN message out of the NameNode:
{code}
12/07/19 10:37:37 WARN ipc.Server: Ignoring socket shutdown exception
{code}
with no further details. This message isn't useful - it either needs to have
more contextual information (eg what client it's talking about and the
exception ignored), or should be reduced to debug level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7761) Improve performance of raw comparisons
[ https://issues.apache.org/jira/browse/HADOOP-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419560#comment-13419560 ] Scott Carey commented on HADOOP-7761: - Any benchmarks showing that this helps hadoop? Based on results in Avro (AVRO-939) so far, this is much slower for small byte arrays and minor for large arrays. Improve performance of raw comparisons -- Key: HADOOP-7761 URL: https://issues.apache.org/jira/browse/HADOOP-7761 Project: Hadoop Common Issue Type: Improvement Components: io, performance, util Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.23.1 Attachments: hadoop-7761.txt, hadoop-7761.txt, hadoop-7761.txt, hadoop-7761.txt Guava has a nice implementation of lexicographical byte-array comparison that uses sun.misc.Unsafe to compare unsigned byte arrays long-at-a-time. Their benchmarks show it as being 2x more CPU-efficient than the equivalent pure-Java implementation. We can easily integrate this into WritableComparator.compareBytes to improve CPU performance in the shuffle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8609) IPC server logs a useless message when shutting down socket
[
https://issues.apache.org/jira/browse/HADOOP-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419566#comment-13419566
]
Hudson commented on HADOOP-8609:
Integrated in Hadoop-Hdfs-trunk-Commit #2574 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2574/])
HADOOP-8609. IPC server logs a useless message when shutting down socket.
Contributed by Jon Zuanich. (Revision 1363950)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1363950
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
IPC server logs a useless message when shutting down socket
---
Key: HADOOP-8609
URL: https://issues.apache.org/jira/browse/HADOOP-8609
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 2.0.0-alpha
Reporter: Todd Lipcon
Assignee: Jon Zuanich
Labels: newbie
Fix For: 2.2.0-alpha
Attachments: hadoop-8609.txt
I occasionally see this WARN message out of the NameNode:
{code}
12/07/19 10:37:37 WARN ipc.Server: Ignoring socket shutdown exception
{code}
with no further details. This message isn't useful - it either needs to have
more contextual information (eg what client it's talking about and the
exception ignored), or should be reduced to debug level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7761) Improve performance of raw comparisons
[ https://issues.apache.org/jira/browse/HADOOP-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419580#comment-13419580 ] Todd Lipcon commented on HADOOP-7761: - Hey Scott. That's interesting -- I did some initial benchmarks back when this was committed, which showed an improvement, _but_ I was benchmarking several patches at once, not just this one in isolation. Shame on me. Does anyone have time to run a terasort before/after (or simulated terasort workload in isolation) and compare CPU_TIME_MILLIS counters on the map side? Improve performance of raw comparisons -- Key: HADOOP-7761 URL: https://issues.apache.org/jira/browse/HADOOP-7761 Project: Hadoop Common Issue Type: Improvement Components: io, performance, util Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.23.1 Attachments: hadoop-7761.txt, hadoop-7761.txt, hadoop-7761.txt, hadoop-7761.txt Guava has a nice implementation of lexicographical byte-array comparison that uses sun.misc.Unsafe to compare unsigned byte arrays long-at-a-time. Their benchmarks show it as being 2x more CPU-efficient than the equivalent pure-Java implementation. We can easily integrate this into WritableComparator.compareBytes to improve CPU performance in the shuffle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8609) IPC server logs a useless message when shutting down socket
[
https://issues.apache.org/jira/browse/HADOOP-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419638#comment-13419638
]
Hudson commented on HADOOP-8609:
Integrated in Hadoop-Mapreduce-trunk-Commit #2530 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2530/])
HADOOP-8609. IPC server logs a useless message when shutting down socket.
Contributed by Jon Zuanich. (Revision 1363950)
Result = FAILURE
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1363950
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
IPC server logs a useless message when shutting down socket
---
Key: HADOOP-8609
URL: https://issues.apache.org/jira/browse/HADOOP-8609
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 2.0.0-alpha
Reporter: Todd Lipcon
Assignee: Jon Zuanich
Labels: newbie
Fix For: 2.2.0-alpha
Attachments: hadoop-8609.txt
I occasionally see this WARN message out of the NameNode:
{code}
12/07/19 10:37:37 WARN ipc.Server: Ignoring socket shutdown exception
{code}
with no further details. This message isn't useful - it either needs to have
more contextual information (eg what client it's talking about and the
exception ignored), or should be reduced to debug level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7423) Document topology script requirements
[ https://issues.apache.org/jira/browse/HADOOP-7423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HADOOP-7423: Target Version/s: 1.2.0, 2.2.0-alpha Affects Version/s: 1.0.0 2.0.0-alpha Your addition looks good. - In trunk/2.x we've switched the docs to apt, here's the relevant page to update: ./hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm. Since we still have the forest doc in trunk might as well update it too (will be removed when we migrate docs over) as you've done here. - Needs to be updated to reflect HADOOP-7030 - Don't think mapred.cache.task.levels exists anymore Document topology script requirements - Key: HADOOP-7423 URL: https://issues.apache.org/jira/browse/HADOOP-7423 Project: Hadoop Common Issue Type: Improvement Components: documentation Affects Versions: 1.0.0, 2.0.0-alpha Reporter: Eli Collins Assignee: Andy Isaacson Labels: newbie Attachments: hadoop-7423.txt The topology script documentation is cluster_setup.xml is unclear. The topology script: # Only needs to handle IP addresses (not hostnames) # Needs to handle multiple arguments for caching to work effectively We should check in an example script or include an example one in the docs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8603) Test failures with Container .. is running beyond virtual memory limits
[
https://issues.apache.org/jira/browse/HADOOP-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419749#comment-13419749
]
Ilya Katsov commented on HADOOP-8603:
-
This is a platform-specific issue, no additional tests can be added.
Test failures with Container .. is running beyond virtual memory limits
-
Key: HADOOP-8603
URL: https://issues.apache.org/jira/browse/HADOOP-8603
Project: Hadoop Common
Issue Type: Bug
Components: test
Affects Versions: 0.23.3
Environment: CentOS 6.2
Reporter: Ilya Katsov
Labels: test
Attachments: HADOOP-8603-branch-0.23.patch,
HADOOP-8603-branch-0.23_002.patch
Tests
org.apache.hadoop.tools.TestHadoopArchives.{testRelativePath,testPathWithSpaces}
fail with the following message:
{code}
Container [pid=7785,containerID=container_1342495768864_0001_01_01] is
running beyond virtual memory limits. Current usage: 143.6mb of 1.5gb
physical memory used; 3.4gb of 3.1gb virtual memory used. Killing container.
Dump of the process-tree for container_1342495768864_0001_01_01 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 7797 7785 7785 7785 (java) 573 38 3517018112 36421
/usr/java/jdk1.6.0_33/jre/bin/java
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.mapreduce.container.log.dir=/var/lib/jenkins/workspace/Hadoop_gd-branch0.23_integration/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1342495768864_0001/container_1342495768864_0001_01_01
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
-Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster
|- 7785 7101 7785 7785 (bash) 1 1 108605440 332 /bin/bash -c
/usr/java/jdk1.6.0_33/jre/bin/java
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.mapreduce.container.log.dir=/var/lib/jenkins/workspace/Hadoop_gd-branch0.23_integration/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1342495768864_0001/container_1342495768864_0001_01_01
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
-Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster
1/var/lib/jenkins/workspace/Hadoop_gd-branch0.23_integration/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1342495768864_0001/container_1342495768864_0001_01_01/stdout
2/var/lib/jenkins/workspace/Hadoop_gd-branch0.23_integration/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1342495768864_0001/container_1342495768864_0001_01_01/stderr
{code}
Is it related to https://issues.apache.org/jira/browse/MAPREDUCE-3933 ? This
is not a stably reproducible problem, but it seems that adding
MALLOC_ARENA_MAX resolves the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
