[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #3247: HDFS-16146. All three replicas are lost due to not adding a new DataN…
Hexiaoqiao commented on a change in pull request #3247:
URL: https://github.com/apache/hadoop/pull/3247#discussion_r680669607
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java
##
@@ -68,6 +68,9 @@ public void delaySendingAckToUpstream(final String
upstreamAddr)
throws IOException {
}
+ public void delayAckLastPacket() throws IOException {
Review comment:
Please add Javadoc for this new injector method.
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
##
@@ -1492,6 +1492,8 @@ public void run() {
if (lastPacketInBlock) {
// Finalize the block and close the block file
finalizeBlock(startTime);
+/* for test only, no-op in production system */
Review comment:
Just suggest to use `//` annotation for single line code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17542) IPV6 support in Netutils#createSocketAddress
[
https://issues.apache.org/jira/browse/HADOOP-17542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391267#comment-17391267
]
Wei-Chiu Chuang commented on HADOOP-17542:
--
I see that PR is being raised against trunk. Should this be done in a branch
instead? [~hemanthboyina]
> IPV6 support in Netutils#createSocketAddress
> -
>
> Key: HADOOP-17542
> URL: https://issues.apache.org/jira/browse/HADOOP-17542
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 3.1.1
>Reporter: ANANDA G B
>Assignee: Renukaprasad C
>Priority: Minor
> Labels: ipv6, pull-request-available
> Attachments: HADOOP-17542-HADOOP-11890-001.patch, Test Scenarios
> Verified in IPV6 cluster.doc
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently NetUtils#createSocketAddress not supporting if target is IPV6 ip.
> If target is IPV6 ip then it throw "Does not contain a valid host:port
> authority: ".
> This need be support.
> public static InetSocketAddress createSocketAddr(String target,
> int defaultPort,
> String configName,
> boolean useCacheIfPresent) {
> String helpText = "";
> if (configName != null)
> { helpText = " (configuration property '" + configName + "')"; }
> if (target == null)
> { throw new IllegalArgumentException("Target address cannot be null." +
> helpText); }
> target = target.trim();
> boolean hasScheme = target.contains("://");
> URI uri = createURI(target, hasScheme, helpText, useCacheIfPresent);
> String host = uri.getHost();
> int port = uri.getPort();
> if (port == -1)
> { port = defaultPort; }
> String path = uri.getPath();
> if ((host == null) || (port < 0) ||
> (!hasScheme && path != null && !path.isEmpty()))
> { throw new IllegalArgumentException( *"Does not contain a valid host:port
> authority: " + target + helpText* ); }
> return createSocketAddrForHost(host, port);
> }
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17612) Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0
[ https://issues.apache.org/jira/browse/HADOOP-17612?focusedWorklogId=632099=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-632099 ] ASF GitHub Bot logged work on HADOOP-17612: --- Author: ASF GitHub Bot Created on: 01/Aug/21 18:53 Start Date: 01/Aug/21 18:53 Worklog Time Spent: 10m Work Description: virajjasani edited a comment on pull request #3241: URL: https://github.com/apache/hadoop/pull/3241#issuecomment-890567536 Addressed comment by @eolivelli. It seems we have enough full build (~20 hr) QA results with latest revision and verified loaded data from PathChildrenCache (e.g keyCache and tokenCache). FYI @aajisaka @jojochuang. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 632099) Time Spent: 3h 40m (was: 3.5h) > Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 > --- > > Key: HADOOP-17612 > URL: https://issues.apache.org/jira/browse/HADOOP-17612 > Project: Hadoop Common > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Let's upgrade Zookeeper and Curator to 3.6.3 and 5.2.0 respectively. > Curator 5.2 also supports Zookeeper 3.5 servers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani edited a comment on pull request #3241: HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0
virajjasani edited a comment on pull request #3241: URL: https://github.com/apache/hadoop/pull/3241#issuecomment-890567536 Addressed comment by @eolivelli. It seems we have enough full build (~20 hr) QA results with latest revision and verified loaded data from PathChildrenCache (e.g keyCache and tokenCache). FYI @aajisaka @jojochuang. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17612) Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0
[ https://issues.apache.org/jira/browse/HADOOP-17612?focusedWorklogId=632098=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-632098 ] ASF GitHub Bot logged work on HADOOP-17612: --- Author: ASF GitHub Bot Created on: 01/Aug/21 18:41 Start Date: 01/Aug/21 18:41 Worklog Time Spent: 10m Work Description: virajjasani commented on pull request #3241: URL: https://github.com/apache/hadoop/pull/3241#issuecomment-890567536 Addressed comment by @eolivelli. It seems we have enough full build (~20 hr) QA results with latest revision. FYI @aajisaka @jojochuang. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 632098) Time Spent: 3.5h (was: 3h 20m) > Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 > --- > > Key: HADOOP-17612 > URL: https://issues.apache.org/jira/browse/HADOOP-17612 > Project: Hadoop Common > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > Let's upgrade Zookeeper and Curator to 3.6.3 and 5.2.0 respectively. > Curator 5.2 also supports Zookeeper 3.5 servers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #3241: HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0
virajjasani commented on pull request #3241: URL: https://github.com/apache/hadoop/pull/3241#issuecomment-890567536 Addressed comment by @eolivelli. It seems we have enough full build (~20 hr) QA results with latest revision. FYI @aajisaka @jojochuang. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] zhangshuyan0 commented on a change in pull request #3247: HDFS-16146. All three replicas are lost due to not adding a new DataN…
zhangshuyan0 commented on a change in pull request #3247:
URL: https://github.com/apache/hadoop/pull/3247#discussion_r680500371
##
File path:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
##
@@ -1386,19 +1386,11 @@ private void addDatanode2ExistingPipeline() throws
IOException {
* Case 2: Failure in Streaming
* - Append/Create:
*+ transfer RBW
- *
- * Case 3: Failure in Close
- * - Append/Create:
- *+ no transfer, let NameNode replicates the block.
*/
if (!isAppend && lastAckedSeqno < 0
&& stage == BlockConstructionStage.PIPELINE_SETUP_CREATE) {
//no data have been written
return;
-} else if (stage == BlockConstructionStage.PIPELINE_CLOSE
Review comment:
Thank you for your suggestion. I just added a UT to this patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3252: YARN-10829. Follow up: Adding null checks before merging ResourceUsage Report
hadoop-yetus commented on pull request #3252:
URL: https://github.com/apache/hadoop/pull/3252#issuecomment-890477691
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 28s | | trunk passed |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 34s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 51s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 6s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 23s | | the patch passed |
| +1 :green_heart: | compile | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 31s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 72m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3252/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3252 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux da00ebf94f0a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b8889b3c48aea0bb576b8b6de6101ad53c678fdf |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3252/1/testReport/ |
| Max. process+thread count | 745 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3252/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please
[GitHub] [hadoop] akshatb1 commented on pull request #3252: YARN-10829. Follow up: Adding null checks before merging ResourceUsage Report
akshatb1 commented on pull request #3252: URL: https://github.com/apache/hadoop/pull/3252#issuecomment-890467737 @bibinchundatt: Could you kindly help in reviewing this PR? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] akshatb1 opened a new pull request #3252: YARN-10829. Follow up: Adding null checks before merging ResourceUsage Report
akshatb1 opened a new pull request #3252: URL: https://github.com/apache/hadoop/pull/3252 - Adding null checks before merging ResourceUsage Report. Follow up from previous PR: https://github.com/apache/hadoop/pull/3135 - Unit tests are running successfully on dev machine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
