[jira] [Created] (HDFS-10632) DataXceiver to report the length of the block it's receiving
Yongjun Zhang created HDFS-10632:
Summary: DataXceiver to report the length of the block it's
receiving
Key: HDFS-10632
URL: https://issues.apache.org/jira/browse/HDFS-10632
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Yongjun Zhang
In DataXceiver:#writeBlock:
{code}
LOG.info("Receiving " + block + " src: " + remoteAddress + " dest: "
+ localAddress);
{code}
This message is better to report the size of the block its receiving.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-10631) Federation State Store ZooKeeper implementation
Inigo Goiri created HDFS-10631: -- Summary: Federation State Store ZooKeeper implementation Key: HDFS-10631 URL: https://issues.apache.org/jira/browse/HDFS-10631 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Inigo Goiri State Store implementation using ZooKeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-10630) Federation State Store
Inigo Goiri created HDFS-10630: -- Summary: Federation State Store Key: HDFS-10630 URL: https://issues.apache.org/jira/browse/HDFS-10630 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs Reporter: Inigo Goiri Interface to store the federation shared state across Routers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-10629) Federation Router
Inigo Goiri created HDFS-10629:
--
Summary: Federation Router
Key: HDFS-10629
URL: https://issues.apache.org/jira/browse/HDFS-10629
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs
Reporter: Inigo Goiri
Component that routes calls from the clients to the right Namespace. It
implements {{ClientProtocol}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-10628) Log HDFS Balancer exit message to its own log
Jiayi Zhou created HDFS-10628: - Summary: Log HDFS Balancer exit message to its own log Key: HDFS-10628 URL: https://issues.apache.org/jira/browse/HDFS-10628 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jiayi Zhou Assignee: Jiayi Zhou Currently, the exit message is logged to stderr. It would be better if we also log this to Balancer log when people want to figure out why Balancer is aborted. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-10627) Volume Scanner mark a block as "suspect" even if the block sender encounters 'Broken pipe' or 'Connection reset by peer' exception
Rushabh S Shah created HDFS-10627:
-
Summary: Volume Scanner mark a block as "suspect" even if the
block sender encounters 'Broken pipe' or 'Connection reset by peer' exception
Key: HDFS-10627
URL: https://issues.apache.org/jira/browse/HDFS-10627
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Affects Versions: 2.7.0
Reporter: Rushabh S Shah
Assignee: Rushabh S Shah
In the BlockSender code,
{code:title=BlockSender.java|borderStyle=solid}
if (!ioem.startsWith("Broken pipe") && !ioem.startsWith("Connection
reset")) {
LOG.error("BlockSender.sendChunks() exception: ", e);
}
datanode.getBlockScanner().markSuspectBlock(
volumeRef.getVolume().getStorageID(),
block);
{code}
Before HDFS-7686, the block was marked as suspect only if the exception message
doesn't start with Broken pipe or Connection reset.
But after HDFS-7686, the block is marked as corrupt irrespectively of the
exception message.
In one of our datanode, it took approximately a whole day (22 hours) to go
through all the suspect blocks to scan one corrupt block.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/ [Jul 13, 2016 7:07:11 AM] (uma.gangumalla) HDFS-10590: Fix TestReconstructStripedBlocks.testCountLiveReplicas test [Jul 13, 2016 12:28:04 PM] (ozawa) HADOOP-11361. Fix a race condition in [Jul 13, 2016 1:42:35 PM] (rohithsharmaks) YARN-5362. TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail. [Jul 13, 2016 3:45:41 PM] (varunsaxena) YARN-5359. FileSystemTimelineReader/Writer uses unix-specific default [Jul 13, 2016 5:54:15 PM] (vinodkv) YARN-5200. Enhanced "yarn logs" to be able to get a list of containers [Jul 13, 2016 8:02:20 PM] (aajisaka) HADOOP-13366. Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic [Jul 13, 2016 8:13:37 PM] (naganarasimha_gr) YARN-5364. timelineservice modules have indirect dependencies on [Jul 13, 2016 9:52:30 PM] (vinodkv) YARN-5339. Fixed "yarn logs" to fail when a file is passed to -out [Jul 13, 2016 11:37:40 PM] (zhz) HADOOP-13290. Appropriate use of generics in FairCallQueue. Contributed [Jul 14, 2016 2:59:35 AM] (kihwal) HDFS-10617. PendingReconstructionBlocks.size() should be synchronized. [Jul 14, 2016 6:18:10 AM] (rohithsharmaks) YARN-5299. Log Docker run command when container fails. Contributed by -1 overall The following subsystems voted -1: unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.server.namenode.TestCheckpoint hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.client.api.impl.TestYarnClient hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorage hadoop.mapred.TestMRCJCFileOutputCommitter cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/diff-compile-javac-root.txt [172K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/diff-checkstyle-root.txt [16M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/diff-patch-pylint.txt [16K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/whitespace-eol.txt [12M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/diff-javadoc-javadoc-root.txt [2.3M] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [144K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [268K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase-tests.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt [92K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/94/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt [124K] Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-10626) VolumeScanner prints incorrect IOException in reportBadBlocks operation
Yiqun Lin created HDFS-10626:
Summary: VolumeScanner prints incorrect IOException in
reportBadBlocks operation
Key: HDFS-10626
URL: https://issues.apache.org/jira/browse/HDFS-10626
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Yiqun Lin
Assignee: Yiqun Lin
VolumeScanner throws incorrect IOException in {{datanode.reportBadBlocks}}. The
related codes:
{code}
public void handle(ExtendedBlock block, IOException e) {
FsVolumeSpi volume = scanner.volume;
...
try {
scanner.datanode.reportBadBlocks(block, volume);
} catch (IOException ie) {
// This is bad, but not bad enough to shut down the scanner.
LOG.warn("Cannot report bad " + block.getBlockId(), e);
}
}
{code}
The IOException that printed in the log should be {{ie}} rather than {{e}}
which was passed in method {{handle(ExtendedBlock block, IOException e)}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
