[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388418#comment-14388418 ] Hudson commented on HDFS-7748: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #149 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/149/]) HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. (wheat9: rev b80457158daf0dc712fbe5695625cc17d70d4bb4) * hadoop-hdfs-project/hadoop-hdfs/src/main/proto/datatransfer.proto * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PipelineAck.java Addendum for HDFS-7748. (wheat9: rev 0967b1d99d7001cd1d09ebd29b9360f1079410e8) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388438#comment-14388438 ] Hudson commented on HDFS-7748: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #883 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/883/]) HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. (wheat9: rev b80457158daf0dc712fbe5695625cc17d70d4bb4) * hadoop-hdfs-project/hadoop-hdfs/src/main/proto/datatransfer.proto * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PipelineAck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java Addendum for HDFS-7748. (wheat9: rev 0967b1d99d7001cd1d09ebd29b9360f1079410e8) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388631#comment-14388631 ] Hudson commented on HDFS-7748: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #140 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/140/]) HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. (wheat9: rev b80457158daf0dc712fbe5695625cc17d70d4bb4) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PipelineAck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/proto/datatransfer.proto * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java Addendum for HDFS-7748. (wheat9: rev 0967b1d99d7001cd1d09ebd29b9360f1079410e8) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388609#comment-14388609 ] Hudson commented on HDFS-7748: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2081 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2081/]) HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. (wheat9: rev b80457158daf0dc712fbe5695625cc17d70d4bb4) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java * hadoop-hdfs-project/hadoop-hdfs/src/main/proto/datatransfer.proto * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PipelineAck.java Addendum for HDFS-7748. (wheat9: rev 0967b1d99d7001cd1d09ebd29b9360f1079410e8) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389151#comment-14389151 ] Hudson commented on HDFS-7748: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2099 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2099/]) HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. (wheat9: rev b80457158daf0dc712fbe5695625cc17d70d4bb4) * hadoop-hdfs-project/hadoop-hdfs/src/main/proto/datatransfer.proto * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PipelineAck.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java Addendum for HDFS-7748. (wheat9: rev 0967b1d99d7001cd1d09ebd29b9360f1079410e8) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389084#comment-14389084 ] Hudson commented on HDFS-7748: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #149 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/149/]) HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. (wheat9: rev b80457158daf0dc712fbe5695625cc17d70d4bb4) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PipelineAck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * hadoop-hdfs-project/hadoop-hdfs/src/main/proto/datatransfer.proto * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Addendum for HDFS-7748. (wheat9: rev 0967b1d99d7001cd1d09ebd29b9360f1079410e8) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Fix For: 2.7.0 Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387264#comment-14387264 ] Vinod Kumar Vavilapalli commented on HDFS-7748: --- +1 for the trivial addendum. Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387302#comment-14387302 ] Zhijie Shen commented on HDFS-7748: --- Compilation fails on trunk too Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387188#comment-14387188 ] Jing Zhao commented on HDFS-7748: - +1 Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387333#comment-14387333 ] Hudson commented on HDFS-7748: -- FAILURE: Integrated in Hadoop-trunk-Commit #7468 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7468/]) Addendum for HDFS-7748. (wheat9: rev 0967b1d99d7001cd1d09ebd29b9360f1079410e8) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387277#comment-14387277
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708221/HDFS-7748.007-addendum.patch
against trunk revision 2a945d2.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10113//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch,
hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch,
hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch,
hdfs-7748.branch-2.7.006.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387221#comment-14387221 ] Hudson commented on HDFS-7748: -- FAILURE: Integrated in Hadoop-trunk-Commit #7466 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7466/]) HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. (wheat9: rev b80457158daf0dc712fbe5695625cc17d70d4bb4) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDataTransferProtocol.java * hadoop-hdfs-project/hadoop-hdfs/src/main/proto/datatransfer.proto * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PipelineAck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387261#comment-14387261
]
Haohui Mai commented on HDFS-7748:
--
The v7 patch accidentally introduces the wrong imports in
{{TestDataTransferProtocol}}, which breaks the compilation of branch-2. I just
uploaded an addendum patch here.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: HDFS-7748.007-addendum.patch, HDFS-7748.007.patch,
hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch,
hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch,
hdfs-7748.branch-2.7.006.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14385402#comment-14385402 ] Anu Engineer commented on HDFS-7748: +1, looks good --Anu Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384393#comment-14384393 ] Anu Engineer commented on HDFS-7748: patches are submitted. we need to finish the review and get it committed. Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384366#comment-14384366 ] Vinod Kumar Vavilapalli commented on HDFS-7748: --- Bump. Is this close to being done for 2.7? Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384840#comment-14384840
]
Hadoop QA commented on HDFS-7748:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707346/hdfs-7748.006.patch
against trunk revision 05499b1.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10093//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10093//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch,
hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384909#comment-14384909 ] Haohui Mai commented on HDFS-7748: -- I take a quick crack based on Anu's work. The only significant difference is that the code combines the bits into the flags which result in better encoding efficiency in protobuf (2 bytes per flags vs. 5bytes). [~anu], does it look good to you? Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: HDFS-7748.007.patch, hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14385086#comment-14385086
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707924/HDFS-7748.007.patch
against trunk revision 3836ad6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestLeaseRecovery2
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10097//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10097//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: HDFS-7748.007.patch, hdfs-7748.001.patch,
hdfs-7748.002.patch, hdfs-7748.003.patch, hdfs-7748.004.patch,
hdfs-7748.005.patch, hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380866#comment-14380866
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707342/hdfs-7748.branch-2.7.006.patch
against trunk revision fc1031a.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10069//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch,
hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381154#comment-14381154
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707346/hdfs-7748.006.patch
against trunk revision fc1031a.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-hdfs-project/hadoop-hdfs
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10070//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10070//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch,
hdfs-7748.006.patch, hdfs-7748.branch-2.7.006.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378904#comment-14378904
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707052/hdfs-7748.005.patch
against trunk revision 53a28af.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10059//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378997#comment-14378997
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707052/hdfs-7748.005.patch
against trunk revision 53a28af.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10060//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch, hdfs-7748.004.patch, hdfs-7748.005.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372371#comment-14372371
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706048/hdfs-7748.004.patch
against trunk revision 586348e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.tracing.TestTracing
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10010//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10010//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch, hdfs-7748.004.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371796#comment-14371796 ] Anu Engineer commented on HDFS-7748: but would that not kill compatibility with old clients ? It thought that was the point of this JIRA all together ? Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371804#comment-14371804 ] Haohui Mai commented on HDFS-7748: -- bq. but would that not kill compatibility with old clients ? It thought that was the point of this JIRA all together ? Let me clarify my proposal: * In the java implementation: combining Status and ECN flags together to ensure that they are always passed together through the call chains. * Over the wire: setting the Status and ECN flags in separate fields in the protobuf messages. Does it make sense to you? Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch, hdfs-7748.003.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371862#comment-14371862
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12705997/hdfs-7748.003.patch
against trunk revision a6a5aae.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:red}-1 eclipse:eclipse{color}. The patch failed to build with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10004//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10004//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371778#comment-14371778
]
Haohui Mai commented on HDFS-7748:
--
{code}
+// A flags structure which lets us send various
+// information back to the client
+enum Flags {
+ // ECN - Explicit Congestion Notification
+ // This tells the client that we have congestion
+ // in the data pipeline. This will allow ECN aware
+ // clients to back-off gracefully.
+
+ ECN_DISABLED = 0; // ECN information is disabled in configuration
+ ECN_SUPPORTED = 1; // We have no congestion in the data pipeline
+ ECN_SUPPORTED2= 2; // Same thing as above, please see RFC-3168
+ ECN_CONGESTED = 3; // We are experiencing a congestion in the data
pipeline
+}
+
{code}
There might be an issue when listing these numbers as an enum -- protobuf will
create an Enum class, will it hinder forward compatibility?
{code}
+ private final static boolean DFS_ECN_ENABLED;
+ DFS_ECN_ENABLED ? Flags.ECN_SUPPORTED_VALUE : Flags.ECN_DISABLED_VALUE);
- public PipelineAck(long seqno, int[] replies,
- long downstreamAckTimeNanos) {
-ArrayListInteger replyList = Lists.newArrayList();
-for (int r : replies) {
- replyList.add(r);
-}
{code}
I start thinking that it might be better to combine Status and ECN flags
internally --- conceptually they are both parts of the header flags, it makes
sense to always combine them together and pass them along, but to put them into
different fields in the protobuf message.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371675#comment-14371675 ] Chris Nauroth commented on HDFS-7748: - Thanks for the revised patch, Anu. I have just one more minor nitpick. hdfs-site.xml is still in the patch with a whitespace change. Let's remove it from the patch completely. With that, all of my feedback has been addressed, and I'm +1 pending another Jenkins run. I'll wait for [~wheat9] to take another look before committing. Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372062#comment-14372062
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12705975/hdfs-7748.002.patch
against trunk revision 1561231.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.tracing.TestTracing
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10002//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10002//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372149#comment-14372149
]
Haohui Mai commented on HDFS-7748:
--
{code}
- public PipelineAck(long seqno, int[] replies,
- long downstreamAckTimeNanos) {
-ArrayListInteger replyList = Lists.newArrayList();
-for (int r : replies) {
- replyList.add(r);
-}
-proto = PipelineAckProto.newBuilder()
+ public PipelineAck(long seqno, Status[] replies,
+ long downstreamAckTimeNanos,
+ int flag) {
+
+PipelineAckProto.Builder protoBuilder = PipelineAckProto.newBuilder()
.setSeqno(seqno)
- .addAllReply(replyList)
- .setDownstreamAckTimeNanos(downstreamAckTimeNanos)
- .build();
+ .addAllStatus(Arrays.asList(replies))
+ .setDownstreamAckTimeNanos(downstreamAckTimeNanos);
+
+if (flag != Flags.ECN_DISABLED_VALUE) {
+ protoBuilder.setFlags(0,flag);
+}
{code}
Note that the ECN flag needs to be preserved through the whole pipeline --
which means that the array should contain the ECN status for all nodes that are
involved in the pipeline.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch, hdfs-7748.002.patch,
hdfs-7748.003.patch, hdfs-7748.004.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368694#comment-14368694
]
Hadoop QA commented on HDFS-7748:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12705511/hdfs-7748.001.patch
against trunk revision bf6295b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.tracing.TestTracing
org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9968//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9968//console
This message is automatically generated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369852#comment-14369852 ] Chris Nauroth commented on HDFS-7748: - Oops, I meant to say that the indentation needs to be 2 spaces. Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: hdfs-7748.001.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369817#comment-14369817
]
Chris Nauroth commented on HDFS-7748:
-
Hi Anu. This patch looks mostly good to me. Here are a few comments.
# {{BlockReceiver.PacketResponder#run}}: There is a line that calculates a
{{myStatus}} local variable. This had been introduced in HDFS-7270. I think
it's unused now and can be removed.
# datatransfer.proto: There is typo in the comment: configration instead of
configuration. Please indent by 4 spaces. (Sorry to nitpick.)
# hdfs-site.xml: Please remove this file from the next revision of the patch.
(I know you had been doing some local testing with the ECN configuration turned
on.)
I think the original intent of HDFS-7270 was to try to match the semantics of
RFC-3168 as closely as possible. Therefore, the {{ECN}} enum needed 4 possible
values, which were named {{DISABLED}}, {{SUPPORTED}}, {{SUPPORTED2}} and
{{CONGESTED}}. Shall we keep the same values here in the new protobuf enum?
I'm not very familiar with this RFC, so I'll defer to [~wheat9] on what is the
best definition of these values.
{{TestTracing}} fails for me on current trunk even without this patch. All
other test failures appear to be unrelated.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[
https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370128#comment-14370128
]
Haohui Mai commented on HDFS-7748:
--
bq. Shall we keep the same values here in the new protobuf enum
Yes. Please keep the enum matching the RFC.
{code}
+ optional ECN ecn = 4 ;
{code}
Let's rename the field into {{header}} and make it a {{int64}} field. That way
allows us to add more information in the future while maintaining both backward
and forward compatibility.
Separate ECN flags from the Status in the DataTransferPipelineAck
-
Key: HDFS-7748
URL: https://issues.apache.org/jira/browse/HDFS-7748
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Anu Engineer
Priority: Blocker
Attachments: hdfs-7748.001.patch
Prior to the discussions on HDFS-7270, the old clients might fail to talk to
the newer server when ECN is turned on. This jira proposes to separate the
ECN flags in a separate protobuf field to make the ack compatible on both
versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369548#comment-14369548 ] Anu Engineer commented on HDFS-7748: Five of these failed due to java.lang.NoClassDefFoundError: , this is probably due to env. issues. testWriteTraceHooks - test failed due to java.lang.AssertionError: timed out to get expected spans: Timed out waiting for condition. Thread diagnostics: Timestamp: 2015-03-19 08:35:22,818 Neither of these looks like related to my change. Thanks Anu Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Anu Engineer Priority: Blocker Attachments: hdfs-7748.001.patch Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357538#comment-14357538 ] Vinod Kumar Vavilapalli commented on HDFS-7748: --- [~wheat9], should 2.7 stop for this? If so, any plans of making progress here? Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Haohui Mai Priority: Blocker Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7748) Separate ECN flags from the Status in the DataTransferPipelineAck
[ https://issues.apache.org/jira/browse/HDFS-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357626#comment-14357626 ] Haohui Mai commented on HDFS-7748: -- Will take care of it in a day or two. Separate ECN flags from the Status in the DataTransferPipelineAck - Key: HDFS-7748 URL: https://issues.apache.org/jira/browse/HDFS-7748 Project: Hadoop HDFS Issue Type: Bug Reporter: Haohui Mai Assignee: Haohui Mai Priority: Blocker Prior to the discussions on HDFS-7270, the old clients might fail to talk to the newer server when ECN is turned on. This jira proposes to separate the ECN flags in a separate protobuf field to make the ack compatible on both versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
