[GitHub] [hadoop] hadoop-yetus commented on pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
hadoop-yetus commented on pull request #2363:
URL: https://github.com/apache/hadoop/pull/2363#issuecomment-704710737
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 14s | | trunk passed |
| +1 :green_heart: | compile | 0m 49s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 31s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 5s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 29s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 26s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 3s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 31s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 10m 21s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 90m 25s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2363 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 2d4e43bd84ef 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1cfe5916e20 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/4/testReport/ |
| Max. process+thread count | 3341 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/4/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Work logged] (HADOOP-17272) ABFS Streams to support IOStatistics API
[
https://issues.apache.org/jira/browse/HADOOP-17272?focusedWorklogId=496300=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496300
]
ASF GitHub Bot logged work on HADOOP-17272:
---
Author: ASF GitHub Bot
Created on: 07/Oct/20 05:17
Start Date: 07/Oct/20 05:17
Worklog Time Spent: 10m
Work Description: mehakmeet commented on a change in pull request #2353:
URL: https://github.com/apache/hadoop/pull/2353#discussion_r500742812
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsInputStreamStatistics.java
##
@@ -160,7 +159,7 @@ public void testSeekStatistics() throws IOException {
assertEquals("Mismatch in forwardSeekOps value", OPERATIONS,
stats.getForwardSeekOperations());
assertEquals("Mismatch in bytesBackwardsOnSeek value",
- -1 * OPERATIONS * ONE_MB, stats.getBytesBackwardsOnSeek());
+ OPERATIONS * ONE_MB, stats.getNegativeBytesBackwardsOnSeek());
Review comment:
bytesBackwardsOnSeek had negative increments and thus had a negative
value but, since we have a restriction that counters can't increment negative
values, I thought to have a negative of bytesBackwardsOnSeek counter so that we
can have a counter to represent this value just in positive. Looks confusing I
guess, should've come up with better naming :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 496300)
Time Spent: 1h 20m (was: 1h 10m)
> ABFS Streams to support IOStatistics API
> -
>
> Key: HADOOP-17272
> URL: https://issues.apache.org/jira/browse/HADOOP-17272
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Steve Loughran
>Assignee: Mehakmeet Singh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> ABFS input/output streams to support IOStatistics API
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mehakmeet commented on a change in pull request #2353: HADOOP-17272. ABFS Streams to support IOStatistics API
mehakmeet commented on a change in pull request #2353:
URL: https://github.com/apache/hadoop/pull/2353#discussion_r500742812
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsInputStreamStatistics.java
##
@@ -160,7 +159,7 @@ public void testSeekStatistics() throws IOException {
assertEquals("Mismatch in forwardSeekOps value", OPERATIONS,
stats.getForwardSeekOperations());
assertEquals("Mismatch in bytesBackwardsOnSeek value",
- -1 * OPERATIONS * ONE_MB, stats.getBytesBackwardsOnSeek());
+ OPERATIONS * ONE_MB, stats.getNegativeBytesBackwardsOnSeek());
Review comment:
bytesBackwardsOnSeek had negative increments and thus had a negative
value but, since we have a restriction that counters can't increment negative
values, I thought to have a negative of bytesBackwardsOnSeek counter so that we
can have a counter to represent this value just in positive. Looks confusing I
guess, should've come up with better naming :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bshashikant commented on pull request #2355: HDFS-15611. Add list Snapshot command in WebHDFS.
bshashikant commented on pull request #2355: URL: https://github.com/apache/hadoop/pull/2355#issuecomment-704694415 Thanks @szetszwo and @smengcl for the review. The 2 checkstyle issues reported are not addressed for better readability of the code. The unit test failures are not related. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bshashikant merged pull request #2355: HDFS-15611. Add list Snapshot command in WebHDFS.
bshashikant merged pull request #2355: URL: https://github.com/apache/hadoop/pull/2355 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2360: HDFS-15613. RBF: Router FSCK fails after HDFS-14442.
aajisaka commented on pull request #2360: URL: https://github.com/apache/hadoop/pull/2360#issuecomment-704686560 Merged. Thank you @ayushtkn and @goiri This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka merged pull request #2360: HDFS-15613. RBF: Router FSCK fails after HDFS-14442.
aajisaka merged pull request #2360: URL: https://github.com/apache/hadoop/pull/2360 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on a change in pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
ferhui commented on a change in pull request #2363:
URL: https://github.com/apache/hadoop/pull/2363#discussion_r500722958
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpc.java
##
@@ -1901,4 +1903,19 @@ private DFSClient getFileDFSClient(final String path) {
}
return null;
}
+
+ @Test
+ public void testCreateWithCallerContext() throws IOException {
+GenericTestUtils.LogCapturer auditlog =
+GenericTestUtils.LogCapturer.captureLogs(FSNamesystem.auditLog);
+
+// Create a directory via the router
+String dirPath = "/test_dir_with_callercontext";
+FsPermission permission = new FsPermission("705");
+routerProtocol.mkdirs(dirPath, permission, false);
+
+// The audit log should contains "callerContext=clientIp:"
+assertTrue(auditlog.getOutput().contains("callerContext=clientIp:"));
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on a change in pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
ferhui commented on a change in pull request #2363:
URL: https://github.com/apache/hadoop/pull/2363#discussion_r500722838
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -116,10 +118,13 @@
/** Optional perf monitor. */
private final RouterRpcMonitor rpcMonitor;
+ private final Configuration clientConf;
+
/** Pattern to parse a stack trace line. */
private static final Pattern STACK_TRACE_PATTERN =
Pattern.compile("\\tat (.*)\\.(.*)\\((.*):(\\d*)\\)");
+ private static final String CLIENT_IP_STR = "clientIp";
Review comment:
[HDFS-13248](https://issues.apache.org/jira/browse/HDFS-13248) focuses
on the locality, here we focus on audit log, does it make sense?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2366: HDFS-15253 Default checkpoint transfer speed, 50mb per second
hadoop-yetus commented on pull request #2366:
URL: https://github.com/apache/hadoop/pull/2366#issuecomment-704670408
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 16s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 3s | | trunk passed |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 1m 9s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 53s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 5s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 49s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 22s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 3m 10s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 8s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 4s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 1m 4s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 48s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 8s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 16m 0s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 3m 17s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 110m 10s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2366/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 200m 21s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.TestFileChecksum |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2366/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2366 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 26ed9484402b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1cfe5916e20 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2366/1/testReport/ |
| Max. process+thread count | 3060 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console
[GitHub] [hadoop] hadoop-yetus commented on pull request #2365: HDFS-15610 Reduced datanode upgrade/hardlink thread from 12 to 6
hadoop-yetus commented on pull request #2365:
URL: https://github.com/apache/hadoop/pull/2365#issuecomment-704669044
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 8s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 22s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 1m 8s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 54s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 50s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 17s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 3m 19s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 17s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 12s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 4s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 1m 4s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 48s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 11s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 36s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 3m 19s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 115m 28s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2365/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 207m 26s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.datanode.TestBPOfferService |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.TestFileChecksumCompositeCrc |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2365/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2365 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux df8ebce07039 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1cfe5916e20 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2365/1/testReport/
[jira] [Updated] (HADOOP-16990) Update Mockserver
[ https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-16990: - Fix Version/s: 3.4.0 > Update Mockserver > - > > Key: HADOOP-16990 > URL: https://issues.apache.org/jira/browse/HADOOP-16990 > Project: Hadoop Common > Issue Type: Task >Reporter: Wei-Chiu Chuang >Assignee: Attila Doroszlai >Priority: Major > Fix For: 3.4.0 > > Attachments: HADOOP-16990.001.patch > > > We are on Mockserver 3.9.2 which is more than 5 years old. Time to update. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] karthikhw opened a new pull request #2366: HDFS-15253 Default checkpoint transfer speed, 50mb per second
karthikhw opened a new pull request #2366: URL: https://github.com/apache/hadoop/pull/2366 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] karthikhw opened a new pull request #2365: HDFS-15610 Reduced datanode upgrade/hardlink thread from 12 to 6
karthikhw opened a new pull request #2365: URL: https://github.com/apache/hadoop/pull/2365 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17271) S3A statistics to support IOStatistics
[
https://issues.apache.org/jira/browse/HADOOP-17271?focusedWorklogId=496221=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496221
]
ASF GitHub Bot logged work on HADOOP-17271:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 22:21
Start Date: 06/Oct/20 22:21
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2324:
URL: https://github.com/apache/hadoop/pull/2324#issuecomment-704583811
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 33m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 4s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 41 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 48s | | trunk passed |
| +1 :green_heart: | compile | 22m 5s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 19m 6s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 3m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 8s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 58s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 1s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 24s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 5m 25s | | trunk passed |
| -0 :warning: | patch | 1m 47s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 8s | | the patch passed |
| +1 :green_heart: | compile | 21m 18s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 21m 18s | |
root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 0 new + 2048 unchanged -
1 fixed = 2048 total (was 2049) |
| +1 :green_heart: | compile | 19m 0s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 19m 0s | |
root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new + 1941 unchanged -
1 fixed = 1941 total (was 1942) |
| -0 :warning: | checkstyle | 2m 58s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2324/12/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 11 new + 267 unchanged - 25 fixed = 278 total
(was 292) |
| +1 :green_heart: | mvnsite | 3m 29s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 0s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 53s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 35s | | hadoop-common in the patch
passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| +1 :green_heart: | javadoc | 0m 37s | | hadoop-mapreduce-client-core
in the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| +1 :green_heart: | javadoc | 0m 46s | |
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new +
0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | findbugs | 5m 54s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 10m 49s | | hadoop-common in the patch
passed. |
[GitHub] [hadoop] hadoop-yetus commented on pull request #2324: HADOOP-17271. S3A statistics to support IOStatistics
hadoop-yetus commented on pull request #2324:
URL: https://github.com/apache/hadoop/pull/2324#issuecomment-704583811
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 33m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 4s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 41 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 48s | | trunk passed |
| +1 :green_heart: | compile | 22m 5s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 19m 6s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 3m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 8s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 58s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 1s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 24s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 5m 25s | | trunk passed |
| -0 :warning: | patch | 1m 47s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 8s | | the patch passed |
| +1 :green_heart: | compile | 21m 18s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 21m 18s | |
root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 0 new + 2048 unchanged -
1 fixed = 2048 total (was 2049) |
| +1 :green_heart: | compile | 19m 0s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 19m 0s | |
root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new + 1941 unchanged -
1 fixed = 1941 total (was 1942) |
| -0 :warning: | checkstyle | 2m 58s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2324/12/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 11 new + 267 unchanged - 25 fixed = 278 total
(was 292) |
| +1 :green_heart: | mvnsite | 3m 29s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 0s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 53s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 35s | | hadoop-common in the patch
passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| +1 :green_heart: | javadoc | 0m 37s | | hadoop-mapreduce-client-core
in the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| +1 :green_heart: | javadoc | 0m 46s | |
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new +
0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | findbugs | 5m 54s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 10m 49s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 7m 27s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 1m 52s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 59s | | The patch does not
generate ASF License warnings. |
| | | 241m 55s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40
[jira] [Commented] (HADOOP-16990) Update Mockserver
[
https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209198#comment-17209198
]
Wei-Chiu Chuang commented on HADOOP-16990:
--
Committed to trunk. Attila, feel free to post a github PR next time.
The commit is clean in branch-3.3 to branch-3.1, but it has the following
dependency divergence:
{noformat}
Dependency convergence error for
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.1 paths to
dependency are:
+-org.apache.hadoop:hadoop-hdfs-client:3.1.5-SNAPSHOT
+-org.mock-server:mockserver-netty:5.11.1
+-org.mock-server:mockserver-core:5.11.1
+-io.swagger.parser.v3:swagger-parser:2.0.20
+-io.swagger.parser.v3:swagger-parser-v3:2.0.20
+-io.swagger.core.v3:swagger-core:2.1.2
+-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.1
and
+-org.apache.hadoop:hadoop-hdfs-client:3.1.5-SNAPSHOT
+-org.mock-server:mockserver-netty:5.11.1
+-org.mock-server:mockserver-core:5.11.1
+-io.swagger.parser.v3:swagger-parser:2.0.20
+-io.swagger.parser.v3:swagger-parser-v3:2.0.20
+-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.2
{noformat}
> Update Mockserver
> -
>
> Key: HADOOP-16990
> URL: https://issues.apache.org/jira/browse/HADOOP-16990
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Wei-Chiu Chuang
>Assignee: Attila Doroszlai
>Priority: Major
> Attachments: HADOOP-16990.001.patch
>
>
> We are on Mockserver 3.9.2 which is more than 5 years old. Time to update.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16990) Update Mockserver
[ https://issues.apache.org/jira/browse/HADOOP-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209191#comment-17209191 ] Wei-Chiu Chuang commented on HADOOP-16990: -- My bad.. missing this one. +1. > Update Mockserver > - > > Key: HADOOP-16990 > URL: https://issues.apache.org/jira/browse/HADOOP-16990 > Project: Hadoop Common > Issue Type: Task >Reporter: Wei-Chiu Chuang >Assignee: Attila Doroszlai >Priority: Major > Attachments: HADOOP-16990.001.patch > > > We are on Mockserver 3.9.2 which is more than 5 years old. Time to update. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2324: HADOOP-17271. S3A statistics to support IOStatistics
hadoop-yetus commented on pull request #2324:
URL: https://github.com/apache/hadoop/pull/2324#issuecomment-704562880
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 4s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 41 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 35s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 24m 24s | | trunk passed |
| +1 :green_heart: | compile | 19m 44s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 17m 2s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 1s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 52s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 51s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 16s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 52s | | trunk passed |
| -0 :warning: | patch | 1m 37s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 55s | | the patch passed |
| +1 :green_heart: | compile | 19m 3s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 19m 3s | |
root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 0 new + 2048 unchanged -
1 fixed = 2048 total (was 2049) |
| +1 :green_heart: | compile | 17m 8s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 8s | |
root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new + 1941 unchanged -
1 fixed = 1941 total (was 1942) |
| -0 :warning: | checkstyle | 2m 52s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2324/11/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 11 new + 267 unchanged - 25 fixed = 278 total
(was 292) |
| +1 :green_heart: | mvnsite | 3m 12s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 21s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 53s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 33s | | hadoop-common in the patch
passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| +1 :green_heart: | javadoc | 0m 35s | | hadoop-mapreduce-client-core
in the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| +1 :green_heart: | javadoc | 0m 43s | |
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new +
0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | findbugs | 5m 15s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 9m 49s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 7m 1s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 1m 43s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 56s | | The patch does not
generate ASF License warnings. |
| | | 194m 44s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40
[jira] [Work logged] (HADOOP-17271) S3A statistics to support IOStatistics
[
https://issues.apache.org/jira/browse/HADOOP-17271?focusedWorklogId=496200=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496200
]
ASF GitHub Bot logged work on HADOOP-17271:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 21:28
Start Date: 06/Oct/20 21:28
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2324:
URL: https://github.com/apache/hadoop/pull/2324#issuecomment-704562880
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 4s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 41 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 35s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 24m 24s | | trunk passed |
| +1 :green_heart: | compile | 19m 44s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 17m 2s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 1s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 52s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 51s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 16s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 52s | | trunk passed |
| -0 :warning: | patch | 1m 37s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 55s | | the patch passed |
| +1 :green_heart: | compile | 19m 3s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 19m 3s | |
root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 0 new + 2048 unchanged -
1 fixed = 2048 total (was 2049) |
| +1 :green_heart: | compile | 17m 8s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 8s | |
root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new + 1941 unchanged -
1 fixed = 1941 total (was 1942) |
| -0 :warning: | checkstyle | 2m 52s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2324/11/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 11 new + 267 unchanged - 25 fixed = 278 total
(was 292) |
| +1 :green_heart: | mvnsite | 3m 12s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 21s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 53s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 33s | | hadoop-common in the patch
passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| +1 :green_heart: | javadoc | 0m 35s | | hadoop-mapreduce-client-core
in the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| +1 :green_heart: | javadoc | 0m 43s | |
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new +
0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | findbugs | 5m 15s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 9m 49s | | hadoop-common in the patch
passed. |
[jira] [Work logged] (HADOOP-16830) Add public IOStatistics API
[
https://issues.apache.org/jira/browse/HADOOP-16830?focusedWorklogId=496177=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496177
]
ASF GitHub Bot logged work on HADOOP-16830:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 20:59
Start Date: 06/Oct/20 20:59
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2323:
URL: https://github.com/apache/hadoop/pull/2323#issuecomment-704549467
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 11 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 28m 46s | | trunk passed |
| +1 :green_heart: | compile | 19m 34s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 16m 50s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 1s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 18s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 16s | | trunk passed |
| -0 :warning: | patch | 2m 38s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | | the patch passed |
| +1 :green_heart: | compile | 18m 53s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| -1 :x: | javac | 18m 53s |
[/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2323/8/artifact/out/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
| root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 20 new + 2050 unchanged
- 0 fixed = 2070 total (was 2050) |
| +1 :green_heart: | compile | 16m 47s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| -1 :x: | javac | 16m 47s |
[/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2323/8/artifact/out/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt)
| root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 20 new + 1946
unchanged - 0 fixed = 1966 total (was 1946) |
| -0 :warning: | checkstyle | 0m 57s |
[/diff-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2323/8/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 5 new + 141
unchanged - 4 fixed = 146 total (was 145) |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 6s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 2m 21s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 9m 46s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 55s | | The patch does not
generate ASF License warnings. |
| | | 160m 36s | | |
[GitHub] [hadoop] hadoop-yetus commented on pull request #2323: HADOOP-16830. Add public IOStatistics API.
hadoop-yetus commented on pull request #2323:
URL: https://github.com/apache/hadoop/pull/2323#issuecomment-704549467
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 11 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 28m 46s | | trunk passed |
| +1 :green_heart: | compile | 19m 34s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 16m 50s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 1s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 18s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 16s | | trunk passed |
| -0 :warning: | patch | 2m 38s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | | the patch passed |
| +1 :green_heart: | compile | 18m 53s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| -1 :x: | javac | 18m 53s |
[/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2323/8/artifact/out/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
| root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 20 new + 2050 unchanged
- 0 fixed = 2070 total (was 2050) |
| +1 :green_heart: | compile | 16m 47s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| -1 :x: | javac | 16m 47s |
[/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2323/8/artifact/out/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt)
| root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 20 new + 1946
unchanged - 0 fixed = 1966 total (was 1946) |
| -0 :warning: | checkstyle | 0m 57s |
[/diff-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2323/8/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 5 new + 141
unchanged - 4 fixed = 146 total (was 145) |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 6s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 2m 21s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 9m 46s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 55s | | The patch does not
generate ASF License warnings. |
| | | 160m 36s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2323/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2323 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle markdownlint |
| uname |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=496146=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496146
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 19:56
Start Date: 06/Oct/20 19:56
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704518289
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 28m 24s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 29m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 9s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 15m 33s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 17s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 33s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 92m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/16/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux 743a91cf2e00 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b361f29dda0 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/16/testReport/ |
| Max. process+thread count | 475 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/16/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 496146)
Time Spent: 6h 10m (was: 6h)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace
[GitHub] [hadoop] hadoop-yetus commented on pull request #2278: HADOOP-17191. ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704518289
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 28m 24s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 29m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 9s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 15m 33s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 17s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 33s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 92m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/16/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux 743a91cf2e00 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b361f29dda0 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/16/testReport/ |
| Max. process+thread count | 475 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/16/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2278: HADOOP-17191. ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704504047
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 51s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 16m 11s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 68m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux 822fade534f1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b361f29dda0 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/15/testReport/ |
| Max. process+thread count | 311 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/15/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=496129=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496129
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 19:27
Start Date: 06/Oct/20 19:27
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704504047
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 51s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 16m 11s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 29s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 68m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint xml |
| uname | Linux 822fade534f1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b361f29dda0 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/15/testReport/ |
| Max. process+thread count | 311 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/15/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 496129)
Time Spent: 6h (was: 5h 50m)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 6h
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace
[GitHub] [hadoop] goiri commented on a change in pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
goiri commented on a change in pull request #2363:
URL: https://github.com/apache/hadoop/pull/2363#discussion_r500540849
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -116,10 +118,13 @@
/** Optional perf monitor. */
private final RouterRpcMonitor rpcMonitor;
+ private final Configuration clientConf;
+
/** Pattern to parse a stack trace line. */
private static final Pattern STACK_TRACE_PATTERN =
Pattern.compile("\\tat (.*)\\.(.*)\\((.*):(\\d*)\\)");
+ private static final String CLIENT_IP_STR = "clientIp";
Review comment:
One of the things that was brought up in the past was to send back in
the context which was the namespace that executed this.
Anyway, let's focus on the locality one for now.
Does it make sense to add a test to make sure we can do proper locality from
the Namenode side with this change? Or should we leave this for the future.
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpc.java
##
@@ -1901,4 +1903,19 @@ private DFSClient getFileDFSClient(final String path) {
}
return null;
}
+
+ @Test
+ public void testCreateWithCallerContext() throws IOException {
+GenericTestUtils.LogCapturer auditlog =
+GenericTestUtils.LogCapturer.captureLogs(FSNamesystem.auditLog);
+
+// Create a directory via the router
+String dirPath = "/test_dir_with_callercontext";
+FsPermission permission = new FsPermission("705");
+routerProtocol.mkdirs(dirPath, permission, false);
+
+// The audit log should contains "callerContext=clientIp:"
+assertTrue(auditlog.getOutput().contains("callerContext=clientIp:"));
Review comment:
I was referring to having this test with a null context but also a test
with some already passed context.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17281) Implement FileSystem.listStatusIterator() in S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-17281?focusedWorklogId=496114=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496114
]
ASF GitHub Bot logged work on HADOOP-17281:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 19:02
Start Date: 06/Oct/20 19:02
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2354:
URL: https://github.com/apache/hadoop/pull/2354#issuecomment-704489524
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 30s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 50s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 37m 43s | | trunk passed |
| +1 :green_heart: | compile | 39m 37s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 33m 4s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 4m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 48s | | trunk passed |
| +1 :green_heart: | shadedclient | 31m 53s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 47s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 56s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 9m 40s | | trunk passed |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 36s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 33s | | the patch passed |
| +1 :green_heart: | compile | 37m 56s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 37m 56s | | the patch passed |
| +1 :green_heart: | compile | 33m 25s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 33m 25s | | the patch passed |
| -0 :warning: | checkstyle | 4m 50s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/4/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 2 new + 112 unchanged - 0 fixed = 114 total (was
112) |
| +1 :green_heart: | mvnsite | 4m 57s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 22m 10s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 41s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 5m 25s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 12m 55s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 16m 0s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 10s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 2m 29s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 34s | | The patch does not
generate ASF License warnings. |
| | | 323m 55s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2354 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle markdownlint |
| uname | Linux c0562df480cc 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
|
[GitHub] [hadoop] hadoop-yetus commented on pull request #2354: HADOOP-17281 Implement FileSystem.listStatusIterator() in S3AFileSystem
hadoop-yetus commented on pull request #2354:
URL: https://github.com/apache/hadoop/pull/2354#issuecomment-704489524
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 30s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 50s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 37m 43s | | trunk passed |
| +1 :green_heart: | compile | 39m 37s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 33m 4s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 4m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 48s | | trunk passed |
| +1 :green_heart: | shadedclient | 31m 53s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 47s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 56s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 9m 40s | | trunk passed |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 36s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 33s | | the patch passed |
| +1 :green_heart: | compile | 37m 56s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 37m 56s | | the patch passed |
| +1 :green_heart: | compile | 33m 25s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 33m 25s | | the patch passed |
| -0 :warning: | checkstyle | 4m 50s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/4/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 2 new + 112 unchanged - 0 fixed = 114 total (was
112) |
| +1 :green_heart: | mvnsite | 4m 57s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 22m 10s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 41s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 5m 25s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 12m 55s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 16m 0s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 10s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 2m 29s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 34s | | The patch does not
generate ASF License warnings. |
| | | 323m 55s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2354 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle markdownlint |
| uname | Linux c0562df480cc 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/4/testReport/ |
| Max. process+thread count | 3117 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
[GitHub] [hadoop] hadoop-yetus commented on pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
hadoop-yetus commented on pull request #2363:
URL: https://github.com/apache/hadoop/pull/2363#issuecomment-704482782
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 25s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 3s | | trunk passed |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 49s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 33s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 48s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 41s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 39s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 44s | | the patch passed |
| +1 :green_heart: | compile | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 47s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| -0 :warning: | checkstyle | 0m 21s |
[/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/3/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 41s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 17m 28s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 6s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 51s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 13m 59s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 43s | | The patch does not
generate ASF License warnings. |
| | | 106m 3s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterRpc |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2363 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 6d1d7495d180 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c9ea344f984 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/3/testReport/ |
| Max. process+thread count | 2760 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=496092=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496092
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 18:22
Start Date: 06/Oct/20 18:22
Worklog Time Spent: 10m
Work Description: bilaharith commented on a change in pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#discussion_r500506248
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -935,3 +935,30 @@ http[s]://[account][domain-suffix]/[filesystem], please
use the following:
{IP}:{PORT}
```
+
+##Run different combinations of tests using the runtests.sh script
+
+This is the expected way in which the tests have to be ran before raising a PR.
+The script runtests.sh contain template for 3 combinations of tests. Ensure
+the auth configs for all the accounts used for testing are provided in
+abfs-testrun-configs.xml. In case any new flags or properties are introduced
+with the code change, add the combinations with the possible configurations
+into the runtests.sh. The thread count can be specified as the command line
+argument for the script. By default the same will be 8.
+
+Adding a combination of tests involves setting the variable scenario (ex: HNS
+-OAuth) and specifying the specific configurations for the particular
+combination with 2 arrays namely properties and values. Specify the property
+names within the array properties and corresponding values in the values
+array. The property and value is determined by the array index. The value for
+the property mentioned at index 1 of array properties should be specified at
+index 1 of the array values. Call the function runtestwithconfs once the 3
+values mentioned are set. Now the script runtests.sh is ready to be ran.
+
+Once the tests are completed, logs will be present in the directory
+target/testlogs. A consolidated test results will be present in the file
+Test-$starttime-Results.log, $startname will be the start time of the test.
+Similarly, the full test report can be found in individual log files, for each
+of the scenarios with the file name Test-$starttime-Logs-$scenario. Please
+attach the consolidates test results from the file Test-$starttime-Results.log
Review comment:
Done
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -935,3 +935,30 @@ http[s]://[account][domain-suffix]/[filesystem], please
use the following:
{IP}:{PORT}
```
+
+##Run different combinations of tests using the runtests.sh script
+
+This is the expected way in which the tests have to be ran before raising a PR.
+The script runtests.sh contain template for 3 combinations of tests. Ensure
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 496092)
Time Spent: 5h 50m (was: 5h 40m)
> ABFS: Run the integration tests with various combinations of configurations
> and publish a consolidated results
> --
>
> Key: HADOOP-17191
> URL: https://issues.apache.org/jira/browse/HADOOP-17191
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Bilahari T H
>Assignee: Bilahari T H
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> ADLS Gen 2 supports accounts with and without hierarchical namespace support.
> ABFS driver supports various authorization mechanisms like OAuth, haredKey,
> Shared Access Signature. The integration tests need to be executed against
> accounts with and without hierarchical namespace support using various
> authorization mechanisms.
> Currently the developer has to manually run the tests with different
> combinations of configurations ex: HNS account with SharedKey and OAuth,
> NonHNS account with SharedKey etc..
> The expectation is to automate these runs with different combinations. This
> will help the developer to run the integration tests with different variants
> of configurations automatically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional
[GitHub] [hadoop] bilaharith commented on a change in pull request #2278: HADOOP-17191. ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
bilaharith commented on a change in pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#discussion_r500506248
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -935,3 +935,30 @@ http[s]://[account][domain-suffix]/[filesystem], please
use the following:
{IP}:{PORT}
```
+
+##Run different combinations of tests using the runtests.sh script
+
+This is the expected way in which the tests have to be ran before raising a PR.
+The script runtests.sh contain template for 3 combinations of tests. Ensure
+the auth configs for all the accounts used for testing are provided in
+abfs-testrun-configs.xml. In case any new flags or properties are introduced
+with the code change, add the combinations with the possible configurations
+into the runtests.sh. The thread count can be specified as the command line
+argument for the script. By default the same will be 8.
+
+Adding a combination of tests involves setting the variable scenario (ex: HNS
+-OAuth) and specifying the specific configurations for the particular
+combination with 2 arrays namely properties and values. Specify the property
+names within the array properties and corresponding values in the values
+array. The property and value is determined by the array index. The value for
+the property mentioned at index 1 of array properties should be specified at
+index 1 of the array values. Call the function runtestwithconfs once the 3
+values mentioned are set. Now the script runtests.sh is ready to be ran.
+
+Once the tests are completed, logs will be present in the directory
+target/testlogs. A consolidated test results will be present in the file
+Test-$starttime-Results.log, $startname will be the start time of the test.
+Similarly, the full test report can be found in individual log files, for each
+of the scenarios with the file name Test-$starttime-Logs-$scenario. Please
+attach the consolidates test results from the file Test-$starttime-Results.log
Review comment:
Done
##
File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md
##
@@ -935,3 +935,30 @@ http[s]://[account][domain-suffix]/[filesystem], please
use the following:
{IP}:{PORT}
```
+
+##Run different combinations of tests using the runtests.sh script
+
+This is the expected way in which the tests have to be ran before raising a PR.
+The script runtests.sh contain template for 3 combinations of tests. Ensure
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] viirya edited a comment on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
viirya edited a comment on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704433770 Ok, got it. I will update release notes once it is back. Seems I cannot update Hadoop JIRA. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17125) Using snappy-java in SnappyCodec
[ https://issues.apache.org/jira/browse/HADOOP-17125?focusedWorklogId=496065=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496065 ] ASF GitHub Bot logged work on HADOOP-17125: --- Author: ASF GitHub Bot Created on: 06/Oct/20 17:51 Start Date: 06/Oct/20 17:51 Worklog Time Spent: 10m Work Description: viirya edited a comment on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704433770 Ok, got it. I will update release notes once it is back. Seems I cannot update Hadoop JIRA. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 496065) Time Spent: 24h 40m (was: 24.5h) > Using snappy-java in SnappyCodec > > > Key: HADOOP-17125 > URL: https://issues.apache.org/jira/browse/HADOOP-17125 > Project: Hadoop Common > Issue Type: New Feature > Components: common >Affects Versions: 3.3.0 >Reporter: DB Tsai >Priority: Major > Labels: pull-request-available > Time Spent: 24h 40m > Remaining Estimate: 0h > > In Hadoop, we use native libs for snappy codec which has several > disadvantages: > * It requires native *libhadoop* and *libsnappy* to be installed in system > *LD_LIBRARY_PATH*, and they have to be installed separately on each node of > the clusters, container images, or local test environments which adds huge > complexities from deployment point of view. In some environments, it requires > compiling the natives from sources which is non-trivial. Also, this approach > is platform dependent; the binary may not work in different platform, so it > requires recompilation. > * It requires extra configuration of *java.library.path* to load the > natives, and it results higher application deployment and maintenance cost > for users. > Projects such as *Spark* and *Parquet* use > [snappy-java|[https://github.com/xerial/snappy-java]] which is JNI-based > implementation. It contains native binaries for Linux, Mac, and IBM in jar > file, and it can automatically load the native binaries into JVM from jar > without any setup. If a native implementation can not be found for a > platform, it can fallback to pure-java implementation of snappy based on > [aircompressor|[https://github.com/airlift/aircompressor/tree/master/src/main/java/io/airlift/compress/snappy]]. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] viirya commented on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
viirya commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704434440 Looks like the JIRA is back now? https://issues.apache.org/jira/browse/HADOOP-17125 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17125) Using snappy-java in SnappyCodec
[ https://issues.apache.org/jira/browse/HADOOP-17125?focusedWorklogId=496050=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496050 ] ASF GitHub Bot logged work on HADOOP-17125: --- Author: ASF GitHub Bot Created on: 06/Oct/20 17:34 Start Date: 06/Oct/20 17:34 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704434440 Looks like the JIRA is back now? https://issues.apache.org/jira/browse/HADOOP-17125 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 496050) Time Spent: 24.5h (was: 24h 20m) > Using snappy-java in SnappyCodec > > > Key: HADOOP-17125 > URL: https://issues.apache.org/jira/browse/HADOOP-17125 > Project: Hadoop Common > Issue Type: New Feature > Components: common >Affects Versions: 3.3.0 >Reporter: DB Tsai >Priority: Major > Labels: pull-request-available > Time Spent: 24.5h > Remaining Estimate: 0h > > In Hadoop, we use native libs for snappy codec which has several > disadvantages: > * It requires native *libhadoop* and *libsnappy* to be installed in system > *LD_LIBRARY_PATH*, and they have to be installed separately on each node of > the clusters, container images, or local test environments which adds huge > complexities from deployment point of view. In some environments, it requires > compiling the natives from sources which is non-trivial. Also, this approach > is platform dependent; the binary may not work in different platform, so it > requires recompilation. > * It requires extra configuration of *java.library.path* to load the > natives, and it results higher application deployment and maintenance cost > for users. > Projects such as *Spark* and *Parquet* use > [snappy-java|[https://github.com/xerial/snappy-java]] which is JNI-based > implementation. It contains native binaries for Linux, Mac, and IBM in jar > file, and it can automatically load the native binaries into JVM from jar > without any setup. If a native implementation can not be found for a > platform, it can fallback to pure-java implementation of snappy based on > [aircompressor|[https://github.com/airlift/aircompressor/tree/master/src/main/java/io/airlift/compress/snappy]]. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] viirya commented on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
viirya commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704433770 Ok, got it. I will update release notes once it is back. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17125) Using snappy-java in SnappyCodec
[ https://issues.apache.org/jira/browse/HADOOP-17125?focusedWorklogId=496049=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496049 ] ASF GitHub Bot logged work on HADOOP-17125: --- Author: ASF GitHub Bot Created on: 06/Oct/20 17:32 Start Date: 06/Oct/20 17:32 Worklog Time Spent: 10m Work Description: viirya commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704433770 Ok, got it. I will update release notes once it is back. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 496049) Time Spent: 24h 20m (was: 24h 10m) > Using snappy-java in SnappyCodec > > > Key: HADOOP-17125 > URL: https://issues.apache.org/jira/browse/HADOOP-17125 > Project: Hadoop Common > Issue Type: New Feature > Components: common >Affects Versions: 3.3.0 >Reporter: DB Tsai >Priority: Major > Labels: pull-request-available > Time Spent: 24h 20m > Remaining Estimate: 0h > > In Hadoop, we use native libs for snappy codec which has several > disadvantages: > * It requires native *libhadoop* and *libsnappy* to be installed in system > *LD_LIBRARY_PATH*, and they have to be installed separately on each node of > the clusters, container images, or local test environments which adds huge > complexities from deployment point of view. In some environments, it requires > compiling the natives from sources which is non-trivial. Also, this approach > is platform dependent; the binary may not work in different platform, so it > requires recompilation. > * It requires extra configuration of *java.library.path* to load the > natives, and it results higher application deployment and maintenance cost > for users. > Projects such as *Spark* and *Parquet* use > [snappy-java|[https://github.com/xerial/snappy-java]] which is JNI-based > implementation. It contains native binaries for Linux, Mac, and IBM in jar > file, and it can automatically load the native binaries into JVM from jar > without any setup. If a native implementation can not be found for a > platform, it can fallback to pure-java implementation of snappy based on > [aircompressor|[https://github.com/airlift/aircompressor/tree/master/src/main/java/io/airlift/compress/snappy]]. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17272) ABFS Streams to support IOStatistics API
[
https://issues.apache.org/jira/browse/HADOOP-17272?focusedWorklogId=496041=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496041
]
ASF GitHub Bot logged work on HADOOP-17272:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 17:18
Start Date: 06/Oct/20 17:18
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#2353:
URL: https://github.com/apache/hadoop/pull/2353#discussion_r500464223
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java
##
@@ -279,7 +289,15 @@ int readRemote(long position, byte[] b, int offset, int
length) throws IOExcepti
AbfsPerfTracker tracker = client.getAbfsPerfTracker();
try (AbfsPerfInfo perfInfo = new AbfsPerfInfo(tracker, "readRemote",
"read")) {
LOG.trace("Trigger client.read for path={} position={} offset={}
length={}", path, position, offset, length);
- op = client.read(path, position, b, offset, length, tolerateOobAppends ?
"*" : eTag, cachedSasToken.get());
+ if (ioStatistics != null) {
+op = IOStatisticsBinding.trackDuration((IOStatisticsStore)
ioStatistics,
Review comment:
I'm going to make trackDuration actually apply the operation, have a
different method to create a wrapper callable
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStreamStatisticsImpl.java
##
@@ -161,55 +170,92 @@ public void remoteBytesRead(long bytes) {
*/
@Override
public void remoteReadOperation() {
-remoteReadOperations++;
+ioStatisticsStore.incrementCounter(getStatName(REMOTE_READ_OP));
}
+ /**
+ * Getter for IOStatistics instance used.
+ * @return IOStatisticsStore instance which extends IOStatistics.
+ */
+ @Override
+ public IOStatistics getIOStatistics() {
+return ioStatisticsStore;
+ }
+
+ @VisibleForTesting
public long getSeekOperations() {
-return seekOperations;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_SEEK_OPERATIONS);
}
+ @VisibleForTesting
public long getForwardSeekOperations() {
-return forwardSeekOperations;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_SEEK_FORWARD_OPERATIONS);
}
+ @VisibleForTesting
public long getBackwardSeekOperations() {
-return backwardSeekOperations;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_SEEK_BACKWARD_OPERATIONS);
}
+ @VisibleForTesting
public long getBytesRead() {
-return bytesRead;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_BYTES);
}
+ @VisibleForTesting
public long getBytesSkippedOnSeek() {
-return bytesSkippedOnSeek;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_SEEK_BYTES_SKIPPED);
}
- public long getBytesBackwardsOnSeek() {
-return bytesBackwardsOnSeek;
+ @VisibleForTesting
+ public long getNegativeBytesBackwardsOnSeek() {
+return
ioStatisticsStore.counters().get(getStatName(NEGATIVE_SEEK_BYTES_BACKWARDS));
}
+ @VisibleForTesting
public long getSeekInBuffer() {
-return seekInBuffer;
+return ioStatisticsStore.counters().get(getStatName(SEEK_IN_BUFFER));
+
}
+ @VisibleForTesting
public long getReadOperations() {
-return readOperations;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_OPERATIONS);
Review comment:
consider static imports for the statistic names
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsInputStreamStatistics.java
##
@@ -160,7 +159,7 @@ public void testSeekStatistics() throws IOException {
assertEquals("Mismatch in forwardSeekOps value", OPERATIONS,
stats.getForwardSeekOperations());
assertEquals("Mismatch in bytesBackwardsOnSeek value",
- -1 * OPERATIONS * ONE_MB, stats.getBytesBackwardsOnSeek());
+ OPERATIONS * ONE_MB, stats.getNegativeBytesBackwardsOnSeek());
Review comment:
why the name change? What does this statistic mean?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 496041)
Time Spent: 1h 10m (was: 1h)
> ABFS Streams to support IOStatistics API
> -
>
> Key: HADOOP-17272
> URL: https://issues.apache.org/jira/browse/HADOOP-17272
> Project: Hadoop Common
> Issue
[GitHub] [hadoop] steveloughran commented on a change in pull request #2353: HADOOP-17272. ABFS Streams to support IOStatistics API
steveloughran commented on a change in pull request #2353:
URL: https://github.com/apache/hadoop/pull/2353#discussion_r500464223
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java
##
@@ -279,7 +289,15 @@ int readRemote(long position, byte[] b, int offset, int
length) throws IOExcepti
AbfsPerfTracker tracker = client.getAbfsPerfTracker();
try (AbfsPerfInfo perfInfo = new AbfsPerfInfo(tracker, "readRemote",
"read")) {
LOG.trace("Trigger client.read for path={} position={} offset={}
length={}", path, position, offset, length);
- op = client.read(path, position, b, offset, length, tolerateOobAppends ?
"*" : eTag, cachedSasToken.get());
+ if (ioStatistics != null) {
+op = IOStatisticsBinding.trackDuration((IOStatisticsStore)
ioStatistics,
Review comment:
I'm going to make trackDuration actually apply the operation, have a
different method to create a wrapper callable
##
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStreamStatisticsImpl.java
##
@@ -161,55 +170,92 @@ public void remoteBytesRead(long bytes) {
*/
@Override
public void remoteReadOperation() {
-remoteReadOperations++;
+ioStatisticsStore.incrementCounter(getStatName(REMOTE_READ_OP));
}
+ /**
+ * Getter for IOStatistics instance used.
+ * @return IOStatisticsStore instance which extends IOStatistics.
+ */
+ @Override
+ public IOStatistics getIOStatistics() {
+return ioStatisticsStore;
+ }
+
+ @VisibleForTesting
public long getSeekOperations() {
-return seekOperations;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_SEEK_OPERATIONS);
}
+ @VisibleForTesting
public long getForwardSeekOperations() {
-return forwardSeekOperations;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_SEEK_FORWARD_OPERATIONS);
}
+ @VisibleForTesting
public long getBackwardSeekOperations() {
-return backwardSeekOperations;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_SEEK_BACKWARD_OPERATIONS);
}
+ @VisibleForTesting
public long getBytesRead() {
-return bytesRead;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_BYTES);
}
+ @VisibleForTesting
public long getBytesSkippedOnSeek() {
-return bytesSkippedOnSeek;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_SEEK_BYTES_SKIPPED);
}
- public long getBytesBackwardsOnSeek() {
-return bytesBackwardsOnSeek;
+ @VisibleForTesting
+ public long getNegativeBytesBackwardsOnSeek() {
+return
ioStatisticsStore.counters().get(getStatName(NEGATIVE_SEEK_BYTES_BACKWARDS));
}
+ @VisibleForTesting
public long getSeekInBuffer() {
-return seekInBuffer;
+return ioStatisticsStore.counters().get(getStatName(SEEK_IN_BUFFER));
+
}
+ @VisibleForTesting
public long getReadOperations() {
-return readOperations;
+return
ioStatisticsStore.counters().get(StreamStatisticNames.STREAM_READ_OPERATIONS);
Review comment:
consider static imports for the statistic names
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsInputStreamStatistics.java
##
@@ -160,7 +159,7 @@ public void testSeekStatistics() throws IOException {
assertEquals("Mismatch in forwardSeekOps value", OPERATIONS,
stats.getForwardSeekOperations());
assertEquals("Mismatch in bytesBackwardsOnSeek value",
- -1 * OPERATIONS * ONE_MB, stats.getBytesBackwardsOnSeek());
+ OPERATIONS * ONE_MB, stats.getNegativeBytesBackwardsOnSeek());
Review comment:
why the name change? What does this statistic mean?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
hadoop-yetus commented on pull request #2363:
URL: https://github.com/apache/hadoop/pull/2363#issuecomment-704425407
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 6s | | trunk passed |
| +1 :green_heart: | compile | 0m 47s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 23s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 29s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 25s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| -0 :warning: | checkstyle | 0m 20s |
[/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/2/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 37s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 56s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 34s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 10m 11s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 91m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2363 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 6944466a4ff6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/2/testReport/ |
| Max. process+thread count | 3068 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/2/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
[GitHub] [hadoop] ferhui commented on a change in pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
ferhui commented on a change in pull request #2363:
URL: https://github.com/apache/hadoop/pull/2363#discussion_r500454655
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -519,6 +525,21 @@ private Object invokeMethod(
}
}
+ /**
+ * For Tracking which is the actual client address.
+ * It adds key/value (clientIp/"ip") pair to the caller context.
+ */
+ private void appendClientIp2CallerContext() {
+final CallerContext ctx = CallerContext.getCurrent();
+CallerContext.Builder builder;
+String origContext = ctx == null ? null : ctx.getContext();
+byte[] origSignature = ctx == null ? null : ctx.getSignature();
+builder = new CallerContext.Builder(origContext, clientConf);
+builder.append(CLIENT_IP_STR, Server.getRemoteAddress());
+builder.setSignature(origSignature);
+CallerContext.setCurrent(builder.build());
Review comment:
Ok
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -519,6 +525,21 @@ private Object invokeMethod(
}
}
+ /**
+ * For Tracking which is the actual client address.
+ * It adds key/value (clientIp/"ip") pair to the caller context.
+ */
+ private void appendClientIp2CallerContext() {
+final CallerContext ctx = CallerContext.getCurrent();
+CallerContext.Builder builder;
Review comment:
Ok
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpc.java
##
@@ -1901,4 +1903,19 @@ private DFSClient getFileDFSClient(final String path) {
}
return null;
}
+
+ @Test
+ public void testCreateWithCallerContext() throws IOException {
+GenericTestUtils.LogCapturer auditlog =
+GenericTestUtils.LogCapturer.captureLogs(FSNamesystem.auditLog);
+
+// Create a directory via the router
+String dirPath = "/test_dir_with_callercontext";
+FsPermission permission = new FsPermission("705");
+routerProtocol.mkdirs(dirPath, permission, false);
+
+// The audit log should contains "callerContext=clientIp:"
+assertTrue(auditlog.getOutput().contains("callerContext=clientIp:"));
Review comment:
old context is null, Will add a check.
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -404,6 +409,7 @@ private Object invokeMethod(
" with params " + Arrays.deepToString(params) + " from "
+ router.getRouterId());
}
+appendClientIp2CallerContext();
Review comment:
ok
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -116,10 +118,13 @@
/** Optional perf monitor. */
private final RouterRpcMonitor rpcMonitor;
+ private final Configuration clientConf;
+
/** Pattern to parse a stack trace line. */
private static final Pattern STACK_TRACE_PATTERN =
Pattern.compile("\\tat (.*)\\.(.*)\\((.*):(\\d*)\\)");
+ private static final String CLIENT_IP_STR = "clientIp";
Review comment:
Now don't know Whether add anything else except client ip, any thoughts?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dbtsai commented on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
dbtsai commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704399048 Thanks all for helping and pushing this through! This will simplify how people deploy snappy native lib greatly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
steveloughran commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704418866 JIRA on apache is offline & updated -we need to remember to update that, including something in the release notes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a change in pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
goiri commented on a change in pull request #2363:
URL: https://github.com/apache/hadoop/pull/2363#discussion_r500431295
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -519,6 +525,21 @@ private Object invokeMethod(
}
}
+ /**
+ * For Tracking which is the actual client address.
+ * It adds key/value (clientIp/"ip") pair to the caller context.
+ */
+ private void appendClientIp2CallerContext() {
+final CallerContext ctx = CallerContext.getCurrent();
+CallerContext.Builder builder;
+String origContext = ctx == null ? null : ctx.getContext();
+byte[] origSignature = ctx == null ? null : ctx.getSignature();
+builder = new CallerContext.Builder(origContext, clientConf);
+builder.append(CLIENT_IP_STR, Server.getRemoteAddress());
+builder.setSignature(origSignature);
+CallerContext.setCurrent(builder.build());
Review comment:
Extract the builder.build()
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpc.java
##
@@ -1901,4 +1903,19 @@ private DFSClient getFileDFSClient(final String path) {
}
return null;
}
+
+ @Test
+ public void testCreateWithCallerContext() throws IOException {
+GenericTestUtils.LogCapturer auditlog =
+GenericTestUtils.LogCapturer.captureLogs(FSNamesystem.auditLog);
+
+// Create a directory via the router
+String dirPath = "/test_dir_with_callercontext";
+FsPermission permission = new FsPermission("705");
+routerProtocol.mkdirs(dirPath, permission, false);
+
+// The audit log should contains "callerContext=clientIp:"
+assertTrue(auditlog.getOutput().contains("callerContext=clientIp:"));
Review comment:
Can we verify we still contain the old context we had?
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -404,6 +409,7 @@ private Object invokeMethod(
" with params " + Arrays.deepToString(params) + " from "
+ router.getRouterId());
}
+appendClientIp2CallerContext();
Review comment:
I would do it more formal: appendClientIpToCallerContext
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -519,6 +525,21 @@ private Object invokeMethod(
}
}
+ /**
+ * For Tracking which is the actual client address.
+ * It adds key/value (clientIp/"ip") pair to the caller context.
+ */
+ private void appendClientIp2CallerContext() {
+final CallerContext ctx = CallerContext.getCurrent();
+CallerContext.Builder builder;
Review comment:
Do this in the moment you create it in line 537.
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -116,10 +118,13 @@
/** Optional perf monitor. */
private final RouterRpcMonitor rpcMonitor;
+ private final Configuration clientConf;
+
/** Pattern to parse a stack trace line. */
private static final Pattern STACK_TRACE_PATTERN =
Pattern.compile("\\tat (.*)\\.(.*)\\((.*):(\\d*)\\)");
+ private static final String CLIENT_IP_STR = "clientIp";
Review comment:
Is there something else we need to add?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] saintstack commented on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
saintstack commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704393077 Thanks for pushing this through @steveloughran +1 on master and branch-3.3. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran merged pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
steveloughran merged pull request #2297: URL: https://github.com/apache/hadoop/pull/2297 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] viirya commented on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
viirya commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704394123 Thanks @steveloughran @saintstack @sunchao @dbtsai This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17125) Using snappy-java in SnappyCodec
[ https://issues.apache.org/jira/browse/HADOOP-17125?focusedWorklogId=495984=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495984 ] ASF GitHub Bot logged work on HADOOP-17125: --- Author: ASF GitHub Bot Created on: 06/Oct/20 16:00 Start Date: 06/Oct/20 16:00 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704380621 Ok, I'm happy too +1, merging to trunk and branch-3.3 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 495984) Time Spent: 24h 10m (was: 24h) > Using snappy-java in SnappyCodec > > > Key: HADOOP-17125 > URL: https://issues.apache.org/jira/browse/HADOOP-17125 > Project: Hadoop Common > Issue Type: New Feature > Components: common >Affects Versions: 3.3.0 >Reporter: DB Tsai >Priority: Major > Labels: pull-request-available > Time Spent: 24h 10m > Remaining Estimate: 0h > > In Hadoop, we use native libs for snappy codec which has several > disadvantages: > * It requires native *libhadoop* and *libsnappy* to be installed in system > *LD_LIBRARY_PATH*, and they have to be installed separately on each node of > the clusters, container images, or local test environments which adds huge > complexities from deployment point of view. In some environments, it requires > compiling the natives from sources which is non-trivial. Also, this approach > is platform dependent; the binary may not work in different platform, so it > requires recompilation. > * It requires extra configuration of *java.library.path* to load the > natives, and it results higher application deployment and maintenance cost > for users. > Projects such as *Spark* and *Parquet* use > [snappy-java|[https://github.com/xerial/snappy-java]] which is JNI-based > implementation. It contains native binaries for Linux, Mac, and IBM in jar > file, and it can automatically load the native binaries into JVM from jar > without any setup. If a native implementation can not be found for a > platform, it can fallback to pure-java implementation of snappy based on > [aircompressor|[https://github.com/airlift/aircompressor/tree/master/src/main/java/io/airlift/compress/snappy]]. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
steveloughran commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704380621 Ok, I'm happy too +1, merging to trunk and branch-3.3 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
steveloughran commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704379411 > No harm fixing it as part of this patch... add the 'jobTokenPassword' from below in ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/pom.xml I think it's actually some test runner bug, really it should be cleaned up. But we can pull in the patch to shut it up. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17125) Using snappy-java in SnappyCodec
[ https://issues.apache.org/jira/browse/HADOOP-17125?focusedWorklogId=495982=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495982 ] ASF GitHub Bot logged work on HADOOP-17125: --- Author: ASF GitHub Bot Created on: 06/Oct/20 15:58 Start Date: 06/Oct/20 15:58 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2297: URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704379411 > No harm fixing it as part of this patch... add the 'jobTokenPassword' from below in ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/pom.xml I think it's actually some test runner bug, really it should be cleaned up. But we can pull in the patch to shut it up. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 495982) Time Spent: 24h (was: 23h 50m) > Using snappy-java in SnappyCodec > > > Key: HADOOP-17125 > URL: https://issues.apache.org/jira/browse/HADOOP-17125 > Project: Hadoop Common > Issue Type: New Feature > Components: common >Affects Versions: 3.3.0 >Reporter: DB Tsai >Priority: Major > Labels: pull-request-available > Time Spent: 24h > Remaining Estimate: 0h > > In Hadoop, we use native libs for snappy codec which has several > disadvantages: > * It requires native *libhadoop* and *libsnappy* to be installed in system > *LD_LIBRARY_PATH*, and they have to be installed separately on each node of > the clusters, container images, or local test environments which adds huge > complexities from deployment point of view. In some environments, it requires > compiling the natives from sources which is non-trivial. Also, this approach > is platform dependent; the binary may not work in different platform, so it > requires recompilation. > * It requires extra configuration of *java.library.path* to load the > natives, and it results higher application deployment and maintenance cost > for users. > Projects such as *Spark* and *Parquet* use > [snappy-java|[https://github.com/xerial/snappy-java]] which is JNI-based > implementation. It contains native binaries for Linux, Mac, and IBM in jar > file, and it can automatically load the native binaries into JVM from jar > without any setup. If a native implementation can not be found for a > platform, it can fallback to pure-java implementation of snappy based on > [aircompressor|[https://github.com/airlift/aircompressor/tree/master/src/main/java/io/airlift/compress/snappy]]. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
ferhui commented on pull request #2363: URL: https://github.com/apache/hadoop/pull/2363#issuecomment-704372005 Fix checkstyle, failed tests pass locally This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495954=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495954 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Oct/20 15:26 Start Date: 06/Oct/20 15:26 Worklog Time Spent: 10m Work Description: steveloughran commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r500379235 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md ## @@ -80,7 +80,7 @@ tests against mocked storage, which is an in-memory emulation of Azure Storage. The integration tests are designed to test directly against an Azure storage service, and require an account and credentials in order to run. -This is done by creating the file to `src/test/resources/azure-auth-keys.xml` +This is done by creating the file to `src/test/resources/abfs-testrun-configs.xml` Review comment: how will this work with the wasb tests? They don't get run through these shell scripts, do they? ## File path: hadoop-tools/hadoop-azure/src/test/resources/abfs-testrun-configs.xml.template ## @@ -0,0 +1,181 @@ + + + + + + + > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 5h 40m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #2278: HADOOP-17191. ABFS: Run the integration tests with various combinations of configurations and publish a consolidated result
steveloughran commented on a change in pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#discussion_r500379235 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md ## @@ -80,7 +80,7 @@ tests against mocked storage, which is an in-memory emulation of Azure Storage. The integration tests are designed to test directly against an Azure storage service, and require an account and credentials in order to run. -This is done by creating the file to `src/test/resources/azure-auth-keys.xml` +This is done by creating the file to `src/test/resources/abfs-testrun-configs.xml` Review comment: how will this work with the wasb tests? They don't get run through these shell scripts, do they? ## File path: hadoop-tools/hadoop-azure/src/test/resources/abfs-testrun-configs.xml.template ## @@ -0,0 +1,181 @@ + + + + + + +
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495948=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495948 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Oct/20 15:11 Start Date: 06/Oct/20 15:11 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-689816045 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 30m 1s | trunk passed | | +1 :green_heart: | compile | 0m 40s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 38s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 31s | trunk passed | | +1 :green_heart: | mvnsite | 0m 43s | trunk passed | | +1 :green_heart: | shadedclient | 14m 27s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 1m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 3s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 18s | the patch passed | | +1 :green_heart: | mvnsite | 0m 30s | the patch passed | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 14m 25s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 27s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 1m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 35s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 74m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux b09effa10fbb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / e5fe3262702 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/8/testReport/ | | Max. process+thread count |
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2278: HADOOP-17191. ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
hadoop-yetus removed a comment on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-689816045 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 30m 1s | trunk passed | | +1 :green_heart: | compile | 0m 40s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 0m 38s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 31s | trunk passed | | +1 :green_heart: | mvnsite | 0m 43s | trunk passed | | +1 :green_heart: | shadedclient | 14m 27s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 1m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 3s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 18s | the patch passed | | +1 :green_heart: | mvnsite | 0m 30s | the patch passed | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | shelldocs | 0m 17s | The patch generated 0 new + 104 unchanged - 132 fixed = 104 total (was 236) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 14m 25s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 27s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 1m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 35s | hadoop-azure in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 74m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2278 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle xml | | uname | Linux b09effa10fbb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / e5fe3262702 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/8/testReport/ | | Max. process+thread count | 401 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/8/console | | versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 findbugs=4.0.6 | | Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated.
[GitHub] [hadoop] hadoop-yetus commented on pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
hadoop-yetus commented on pull request #2363:
URL: https://github.com/apache/hadoop/pull/2363#issuecomment-704337731
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 45s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 41s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 23s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 21s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| -0 :warning: | checkstyle | 0m 19s |
[/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/1/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 2 new + 0
unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 36s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 30s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 24s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 9m 39s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 86m 8s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.federation.security.token.TestZKDelegationTokenSecretManagerImpl
|
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2363 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 7785ca08b9cb 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2363/1/testReport/ |
| Max. process+thread count | 3599 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495945=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495945 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot Created on: 06/Oct/20 15:09 Start Date: 06/Oct/20 15:09 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704137336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 495945) Time Spent: 5h 20m (was: 5h 10m) > ABFS: Run the integration tests with various combinations of configurations > and publish a consolidated results > -- > > Key: HADOOP-17191 > URL: https://issues.apache.org/jira/browse/HADOOP-17191 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > Labels: pull-request-available > Time Spent: 5h 20m > Remaining Estimate: 0h > > ADLS Gen 2 supports accounts with and without hierarchical namespace support. > ABFS driver supports various authorization mechanisms like OAuth, haredKey, > Shared Access Signature. The integration tests need to be executed against > accounts with and without hierarchical namespace support using various > authorization mechanisms. > Currently the developer has to manually run the tests with different > combinations of configurations ex: HNS account with SharedKey and OAuth, > NonHNS account with SharedKey etc.. > The expectation is to automate these runs with different combinations. This > will help the developer to run the integration tests with different variants > of configurations automatically. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2278: HADOOP-17191. ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
hadoop-yetus removed a comment on pull request #2278: URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704137336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17293) refreshing S3Guard records after TTL-triggered-HEAD breaks some workflows
[ https://issues.apache.org/jira/browse/HADOOP-17293?focusedWorklogId=495943=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495943 ] ASF GitHub Bot logged work on HADOOP-17293: --- Author: ASF GitHub Bot Created on: 06/Oct/20 15:07 Start Date: 06/Oct/20 15:07 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#issuecomment-704335369 tests with markers=keep & delete all good other than * continuous read() under-fulfillment breaking tests which expect their buffers to always be full. Root cause clearly some networking issue, but where? * ITestS3AInputStreamPerformance stream had to be reopened 3 times, rather than 1. Didn't come back on a rerun. ``` [INFO] Running org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationInFileystem [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 386.753 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance [ERROR] testDecompressionSequential128K(org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance) Time elapsed: 215.757 s <<< FAILURE! java.lang.AssertionError: open operations in null expected:<1> but was:<3> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:645) at org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance.assertOpenOperationCount(ITestS3AInputStreamPerformance.java:189) at org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance.assertStreamOpenedExactlyOnce(ITestS3AInputStreamPerformance.java:181) at org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance.testDecompressionSequential128K(ITestS3AInputStreamPerformance.java:324) ``` Assumption: again, network playing up. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 495943) Time Spent: 1h (was: 50m) > refreshing S3Guard records after TTL-triggered-HEAD breaks some workflows > - > > Key: HADOOP-17293 > URL: https://issues.apache.org/jira/browse/HADOOP-17293 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > an incidental part of HDP-13230 was a fix to innerGetFileStatus, wherein > after a HEAD request we would update the DDB record, so resetting it's TTL > Applications which did remote updates of buckets without going through > s3guard are now triggering failures in applications in the cluster when they > go to open the file -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2361: HADOOP-17293. Aways probe S3 when opening/getFileStatus on non-auth paths
steveloughran commented on pull request #2361: URL: https://github.com/apache/hadoop/pull/2361#issuecomment-704335369 tests with markers=keep & delete all good other than * continuous read() under-fulfillment breaking tests which expect their buffers to always be full. Root cause clearly some networking issue, but where? * ITestS3AInputStreamPerformance stream had to be reopened 3 times, rather than 1. Didn't come back on a rerun. ``` [INFO] Running org.apache.hadoop.fs.s3a.auth.delegation.ITestRoleDelegationInFileystem [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 386.753 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance [ERROR] testDecompressionSequential128K(org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance) Time elapsed: 215.757 s <<< FAILURE! java.lang.AssertionError: open operations in null expected:<1> but was:<3> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:645) at org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance.assertOpenOperationCount(ITestS3AInputStreamPerformance.java:189) at org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance.assertStreamOpenedExactlyOnce(ITestS3AInputStreamPerformance.java:181) at org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance.testDecompressionSequential128K(ITestS3AInputStreamPerformance.java:324) ``` Assumption: again, network playing up. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2364: Merge pull request #1 from apache/trunk
hadoop-yetus commented on pull request #2364:
URL: https://github.com/apache/hadoop/pull/2364#issuecomment-704317137
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 47s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 23m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 20s | | Maven dependency ordering for patch |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 17m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 45m 16s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2364/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2364 |
| Optional Tests | dupname asflicense |
| uname | Linux 083366e69ba6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2364/1/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17265) ABFS: Support for Client Correlation ID
[
https://issues.apache.org/jira/browse/HADOOP-17265?focusedWorklogId=495924=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495924
]
ASF GitHub Bot logged work on HADOOP-17265:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 14:35
Start Date: 06/Oct/20 14:35
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2344:
URL: https://github.com/apache/hadoop/pull/2344#issuecomment-704312179
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 52s | | trunk passed |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 33s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 48s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 58s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 56s | | trunk passed |
| -0 :warning: | patch | 1m 13s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 27s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 23s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 54s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 16s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 40s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 78m 50s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2344/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2344 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle markdownlint |
| uname | Linux 385bce5f7d14 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2344/10/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 5500)
[GitHub] [hadoop] hadoop-yetus commented on pull request #2344: HADOOP-17265. ABFS: Support for Client Correlation ID
hadoop-yetus commented on pull request #2344:
URL: https://github.com/apache/hadoop/pull/2344#issuecomment-704312179
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 52s | | trunk passed |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 33s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 48s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 58s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 56s | | trunk passed |
| -0 :warning: | patch | 1m 13s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 27s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 23s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 54s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 16s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 40s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 78m 50s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2344/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2344 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle markdownlint |
| uname | Linux 385bce5f7d14 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2344/10/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2344/10/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2
[ https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208798#comment-17208798 ] Hemanth Boyina commented on HADOOP-17144: - sorry for being late , thanks for the review [~iwasakims] updated the patch , please review > Update Hadoop's lz4 to v1.9.2 > - > > Key: HADOOP-17144 > URL: https://issues.apache.org/jira/browse/HADOOP-17144 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Major > Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, > HADOOP-17144.003.patch, HADOOP-17144.004.patch, HADOOP-17144.005.patch > > > Update hadoop's native lz4 to v1.9.2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2
[ https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina updated HADOOP-17144: Attachment: HADOOP-17144.005.patch > Update Hadoop's lz4 to v1.9.2 > - > > Key: HADOOP-17144 > URL: https://issues.apache.org/jira/browse/HADOOP-17144 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Major > Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, > HADOOP-17144.003.patch, HADOOP-17144.004.patch, HADOOP-17144.005.patch > > > Update hadoop's native lz4 to v1.9.2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17281) Implement FileSystem.listStatusIterator() in S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-17281?focusedWorklogId=495884=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495884
]
ASF GitHub Bot logged work on HADOOP-17281:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 12:59
Start Date: 06/Oct/20 12:59
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#2354:
URL: https://github.com/apache/hadoop/pull/2354#discussion_r500254718
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java
##
@@ -1520,7 +1521,8 @@ public static TreeScanResults treeWalk(FileSystem fs,
Path path)
while (true) {
list.add(iterator.next());
}
-} catch (NoSuchElementException expected) {
+} catch (NoSuchElementException | IllegalStateException expected) {
Review comment:
as discussed: better to fix DirListingIterator to match RemoteIterator
API; we can consider it's failure a bug
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractGetFileStatusTest.java
##
@@ -359,16 +357,39 @@ public void testListStatusFile() throws Throwable {
public void testListStatusIteratorFile() throws Throwable {
describe("test the listStatusIterator(path) on a file");
Path f = touchf("listStItrFile");
+
List statusList = (List) iteratorToList(
getFileSystem().listStatusIterator(f));
-assertEquals("size of file list returned", 1, statusList.size());
-assertIsNamedFile(f, statusList.get(0));
+validateListingForFile(f, statusList, false);
+
List statusList2 =
(List) iteratorToListThroughNextCallsAlone(
getFileSystem().listStatusIterator(f));
-assertEquals("size of file list returned through next() calls",
-1, statusList2.size());
-assertIsNamedFile(f, statusList2.get(0));
+validateListingForFile(f, statusList2, true);
+ }
+
+ /**
+ * Validate listing result for an input path which is file.
+ * @param f file.
+ * @param statusList list status of a file.
+ * @param nextCallAlone whether the listing generated just using
+ * next() calls.
+ */
+ private void validateListingForFile(Path f,
+ List statusList,
+ boolean nextCallAlone) {
+String msg = String.format("size of file list returned using %s should " +
+"be 1", nextCallAlone ?
+"next() calls alone" : "hasNext() and next() calls");
+Assertions.assertThat(statusList)
+.describedAs(msg)
+.hasSize(1);
+Assertions.assertThat(statusList.get(0).getPath().toString())
+.describedAs("path returned should match with the input path")
Review comment:
any reason not to leave both as Path and let Path.equals() to do the
work? It compares URIs
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 495884)
Time Spent: 1.5h (was: 1h 20m)
> Implement FileSystem.listStatusIterator() in S3AFileSystem
> --
>
> Key: HADOOP-17281
> URL: https://issues.apache.org/jira/browse/HADOOP-17281
> Project: Hadoop Common
> Issue Type: Task
> Components: fs/s3
>Affects Versions: 3.3.0
>Reporter: Mukund Thakur
>Assignee: Mukund Thakur
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Currently S3AFileSystem only implements listStatus() api which returns an
> array. Once we implement the listStatusIterator(), clients can benefit from
> the async listing done recently
> https://issues.apache.org/jira/browse/HADOOP-17074 by performing some tasks
> on files while iterating them.
>
> CC [~stevel]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] zml4518079 opened a new pull request #2364: Merge pull request #1 from apache/trunk
zml4518079 opened a new pull request #2364: URL: https://github.com/apache/hadoop/pull/2364 sychronizing code ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17272) ABFS Streams to support IOStatistics API
[
https://issues.apache.org/jira/browse/HADOOP-17272?focusedWorklogId=495918=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495918
]
ASF GitHub Bot logged work on HADOOP-17272:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 13:53
Start Date: 06/Oct/20 13:53
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2353:
URL: https://github.com/apache/hadoop/pull/2353#issuecomment-704284203
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 14 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 32m 53s | | trunk passed |
| +1 :green_heart: | compile | 28m 39s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 23m 46s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 3m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 55s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 34s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 22s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 7s | | trunk passed |
| -0 :warning: | patch | 1m 48s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 48s | | the patch passed |
| +1 :green_heart: | compile | 26m 29s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| -1 :x: | javac | 26m 29s |
[/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2353/3/artifact/out/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
| root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 20 new + 2050 unchanged
- 0 fixed = 2070 total (was 2050) |
| +1 :green_heart: | compile | 22m 28s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| -1 :x: | javac | 22m 28s |
[/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2353/3/artifact/out/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt)
| root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 20 new + 1945
unchanged - 0 fixed = 1965 total (was 1945) |
| -0 :warning: | checkstyle | 3m 14s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2353/3/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 10 new + 141 unchanged - 4 fixed = 151 total (was
145) |
| +1 :green_heart: | mvnsite | 2m 45s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 16m 39s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 36s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 4m 50s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 11m 53s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 5s | | hadoop-azure in the
[GitHub] [hadoop] hadoop-yetus commented on pull request #2353: HADOOP-17272. ABFS Streams to support IOStatistics API
hadoop-yetus commented on pull request #2353:
URL: https://github.com/apache/hadoop/pull/2353#issuecomment-704284203
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 14 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 32m 53s | | trunk passed |
| +1 :green_heart: | compile | 28m 39s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 23m 46s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 3m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 55s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 34s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 22s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 4m 7s | | trunk passed |
| -0 :warning: | patch | 1m 48s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 48s | | the patch passed |
| +1 :green_heart: | compile | 26m 29s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| -1 :x: | javac | 26m 29s |
[/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2353/3/artifact/out/diff-compile-javac-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
| root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 20 new + 2050 unchanged
- 0 fixed = 2070 total (was 2050) |
| +1 :green_heart: | compile | 22m 28s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| -1 :x: | javac | 22m 28s |
[/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2353/3/artifact/out/diff-compile-javac-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt)
| root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 20 new + 1945
unchanged - 0 fixed = 1965 total (was 1945) |
| -0 :warning: | checkstyle | 3m 14s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2353/3/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 10 new + 141 unchanged - 4 fixed = 151 total (was
145) |
| +1 :green_heart: | mvnsite | 2m 45s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 16m 39s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 36s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 4m 50s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 11m 53s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 5s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 4s | | The patch does not
generate ASF License warnings. |
| | | 228m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2353/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2353
[jira] [Updated] (HADOOP-17300) FileSystem.DirListingIterator.next() call should return NoSuchElementException
[
https://issues.apache.org/jira/browse/HADOOP-17300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Thakur updated HADOOP-17300:
---
Description:
FileSystem.DirListingIterator.next() call should return NoSuchElementException
rather than IllegalStateException
Stacktrace for new test failure:
{code:java}
java.lang.IllegalStateException: No more items in
iteratorjava.lang.IllegalStateException: No more items in iterator at
com.google.common.base.Preconditions.checkState(Preconditions.java:507) at
org.apache.hadoop.fs.FileSystem$DirListingIterator.next(FileSystem.java:2232)
at
org.apache.hadoop.fs.FileSystem$DirListingIterator.next(FileSystem.java:2205)
at
org.apache.hadoop.fs.contract.ContractTestUtils.iteratorToListThroughNextCallsAlone(ContractTestUtils.java:1495)
at
org.apache.hadoop.fs.contract.AbstractContractGetFileStatusTest.testListStatusIteratorFile(AbstractContractGetFileStatusTest.java:366)
{code}
CC [~ste...@apache.org]
was:
FileSystem.DirListingIterator.next() call should return NoSuchElementException
rather than IllegalStateException
CC [~ste...@apache.org]
> FileSystem.DirListingIterator.next() call should return NoSuchElementException
> --
>
> Key: HADOOP-17300
> URL: https://issues.apache.org/jira/browse/HADOOP-17300
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, fs
>Reporter: Mukund Thakur
>Assignee: Mukund Thakur
>Priority: Major
>
> FileSystem.DirListingIterator.next() call should return
> NoSuchElementException rather than IllegalStateException
>
> Stacktrace for new test failure:
>
> {code:java}
> java.lang.IllegalStateException: No more items in
> iteratorjava.lang.IllegalStateException: No more items in iterator at
> com.google.common.base.Preconditions.checkState(Preconditions.java:507) at
> org.apache.hadoop.fs.FileSystem$DirListingIterator.next(FileSystem.java:2232)
> at
> org.apache.hadoop.fs.FileSystem$DirListingIterator.next(FileSystem.java:2205)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.iteratorToListThroughNextCallsAlone(ContractTestUtils.java:1495)
> at
> org.apache.hadoop.fs.contract.AbstractContractGetFileStatusTest.testListStatusIteratorFile(AbstractContractGetFileStatusTest.java:366)
> {code}
>
> CC [~ste...@apache.org]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui opened a new pull request #2363: HDFS-13293. RBF: The RouterRPCServer should transfer CallerContext an…
ferhui opened a new pull request #2363: URL: https://github.com/apache/hadoop/pull/2363 …d client ip to NamenodeRpcServer ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17300) FileSystem.DirListingIterator.next() call should return NoSuchElementException
[
https://issues.apache.org/jira/browse/HADOOP-17300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208759#comment-17208759
]
Mukund Thakur commented on HADOOP-17300:
https://github.com/apache/hadoop/pull/2354
> FileSystem.DirListingIterator.next() call should return NoSuchElementException
> --
>
> Key: HADOOP-17300
> URL: https://issues.apache.org/jira/browse/HADOOP-17300
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, fs
>Reporter: Mukund Thakur
>Assignee: Mukund Thakur
>Priority: Major
>
> FileSystem.DirListingIterator.next() call should return
> NoSuchElementException rather than IllegalStateException
>
> Stacktrace for new test failure:
>
> {code:java}
> java.lang.IllegalStateException: No more items in
> iteratorjava.lang.IllegalStateException: No more items in iterator at
> com.google.common.base.Preconditions.checkState(Preconditions.java:507) at
> org.apache.hadoop.fs.FileSystem$DirListingIterator.next(FileSystem.java:2232)
> at
> org.apache.hadoop.fs.FileSystem$DirListingIterator.next(FileSystem.java:2205)
> at
> org.apache.hadoop.fs.contract.ContractTestUtils.iteratorToListThroughNextCallsAlone(ContractTestUtils.java:1495)
> at
> org.apache.hadoop.fs.contract.AbstractContractGetFileStatusTest.testListStatusIteratorFile(AbstractContractGetFileStatusTest.java:366)
> {code}
>
> CC [~ste...@apache.org]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2354: HADOOP-17281 Implement FileSystem.listStatusIterator() in S3AFileSystem
mukund-thakur commented on a change in pull request #2354: URL: https://github.com/apache/hadoop/pull/2354#discussion_r500284497 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java ## @@ -1107,6 +1107,7 @@ public Void next(final FileSystem fs, final Path p) } HdfsFileStatus[] partialListing = thisListing.getPartialListing(); + Review comment: Please remove this before changing. Was just added to trigger hdfs tests by yetus. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17281) Implement FileSystem.listStatusIterator() in S3AFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-17281?focusedWorklogId=495905=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495905 ] ASF GitHub Bot logged work on HADOOP-17281: --- Author: ASF GitHub Bot Created on: 06/Oct/20 13:38 Start Date: 06/Oct/20 13:38 Worklog Time Spent: 10m Work Description: mukund-thakur commented on a change in pull request #2354: URL: https://github.com/apache/hadoop/pull/2354#discussion_r500284497 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java ## @@ -1107,6 +1107,7 @@ public Void next(final FileSystem fs, final Path p) } HdfsFileStatus[] partialListing = thisListing.getPartialListing(); + Review comment: Please remove this before changing. Was just added to trigger hdfs tests by yetus. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 495905) Time Spent: 1h 40m (was: 1.5h) > Implement FileSystem.listStatusIterator() in S3AFileSystem > -- > > Key: HADOOP-17281 > URL: https://issues.apache.org/jira/browse/HADOOP-17281 > Project: Hadoop Common > Issue Type: Task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Currently S3AFileSystem only implements listStatus() api which returns an > array. Once we implement the listStatusIterator(), clients can benefit from > the async listing done recently > https://issues.apache.org/jira/browse/HADOOP-17074 by performing some tasks > on files while iterating them. > > CC [~stevel] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17281) Implement FileSystem.listStatusIterator() in S3AFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-17281?focusedWorklogId=495906=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495906 ] ASF GitHub Bot logged work on HADOOP-17281: --- Author: ASF GitHub Bot Created on: 06/Oct/20 13:38 Start Date: 06/Oct/20 13:38 Worklog Time Spent: 10m Work Description: mukund-thakur commented on a change in pull request #2354: URL: https://github.com/apache/hadoop/pull/2354#discussion_r500284497 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java ## @@ -1107,6 +1107,7 @@ public Void next(final FileSystem fs, final Path p) } HdfsFileStatus[] partialListing = thisListing.getPartialListing(); + Review comment: Please remove this before merging. Was just added to trigger hdfs tests by yetus. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 495906) Time Spent: 1h 50m (was: 1h 40m) > Implement FileSystem.listStatusIterator() in S3AFileSystem > -- > > Key: HADOOP-17281 > URL: https://issues.apache.org/jira/browse/HADOOP-17281 > Project: Hadoop Common > Issue Type: Task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently S3AFileSystem only implements listStatus() api which returns an > array. Once we implement the listStatusIterator(), clients can benefit from > the async listing done recently > https://issues.apache.org/jira/browse/HADOOP-17074 by performing some tasks > on files while iterating them. > > CC [~stevel] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2354: HADOOP-17281 Implement FileSystem.listStatusIterator() in S3AFileSystem
mukund-thakur commented on a change in pull request #2354: URL: https://github.com/apache/hadoop/pull/2354#discussion_r500284497 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java ## @@ -1107,6 +1107,7 @@ public Void next(final FileSystem fs, final Path p) } HdfsFileStatus[] partialListing = thisListing.getPartialListing(); + Review comment: Please remove this before merging. Was just added to trigger hdfs tests by yetus. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #2354: HADOOP-17281 Implement FileSystem.listStatusIterator() in S3AFileSystem
steveloughran commented on a change in pull request #2354:
URL: https://github.com/apache/hadoop/pull/2354#discussion_r500254718
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java
##
@@ -1520,7 +1521,8 @@ public static TreeScanResults treeWalk(FileSystem fs,
Path path)
while (true) {
list.add(iterator.next());
}
-} catch (NoSuchElementException expected) {
+} catch (NoSuchElementException | IllegalStateException expected) {
Review comment:
as discussed: better to fix DirListingIterator to match RemoteIterator
API; we can consider it's failure a bug
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractGetFileStatusTest.java
##
@@ -359,16 +357,39 @@ public void testListStatusFile() throws Throwable {
public void testListStatusIteratorFile() throws Throwable {
describe("test the listStatusIterator(path) on a file");
Path f = touchf("listStItrFile");
+
List statusList = (List) iteratorToList(
getFileSystem().listStatusIterator(f));
-assertEquals("size of file list returned", 1, statusList.size());
-assertIsNamedFile(f, statusList.get(0));
+validateListingForFile(f, statusList, false);
+
List statusList2 =
(List) iteratorToListThroughNextCallsAlone(
getFileSystem().listStatusIterator(f));
-assertEquals("size of file list returned through next() calls",
-1, statusList2.size());
-assertIsNamedFile(f, statusList2.get(0));
+validateListingForFile(f, statusList2, true);
+ }
+
+ /**
+ * Validate listing result for an input path which is file.
+ * @param f file.
+ * @param statusList list status of a file.
+ * @param nextCallAlone whether the listing generated just using
+ * next() calls.
+ */
+ private void validateListingForFile(Path f,
+ List statusList,
+ boolean nextCallAlone) {
+String msg = String.format("size of file list returned using %s should " +
+"be 1", nextCallAlone ?
+"next() calls alone" : "hasNext() and next() calls");
+Assertions.assertThat(statusList)
+.describedAs(msg)
+.hasSize(1);
+Assertions.assertThat(statusList.get(0).getPath().toString())
+.describedAs("path returned should match with the input path")
Review comment:
any reason not to leave both as Path and let Path.equals() to do the
work? It compares URIs
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] eltociear commented on pull request #2357: HADOOP-17294. Fix typos existance to existence
eltociear commented on pull request #2357: URL: https://github.com/apache/hadoop/pull/2357#issuecomment-704245972 @tasanuma Thank you for your reviewing! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17294) Fix typos existance to existence
[ https://issues.apache.org/jira/browse/HADOOP-17294?focusedWorklogId=495881=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495881 ] ASF GitHub Bot logged work on HADOOP-17294: --- Author: ASF GitHub Bot Created on: 06/Oct/20 12:49 Start Date: 06/Oct/20 12:49 Worklog Time Spent: 10m Work Description: eltociear commented on pull request #2357: URL: https://github.com/apache/hadoop/pull/2357#issuecomment-704245972 @tasanuma Thank you for your reviewing! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 495881) Time Spent: 1h 20m (was: 1h 10m) > Fix typos existance to existence > > > Key: HADOOP-17294 > URL: https://issues.apache.org/jira/browse/HADOOP-17294 > Project: Hadoop Common > Issue Type: Bug >Reporter: Ikko Ashimine >Assignee: Ikko Ashimine >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: HADOOP-17294_patch.diff > > Time Spent: 1h 20m > Remaining Estimate: 0h > > existance -> existence -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495878=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495878
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 12:38
Start Date: 06/Oct/20 12:38
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704239734
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 5s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 0s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 59s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 57s | | trunk passed |
| -0 :warning: | patch | 1m 15s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 41s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 0s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 77m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle
xml |
| uname | Linux dcb5b341b388 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK
[GitHub] [hadoop] hadoop-yetus commented on pull request #2278: HADOOP-17191. ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704239734
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 5s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 0s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 59s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 57s | | trunk passed |
| -0 :warning: | patch | 1m 15s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 41s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 0s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 77m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle
xml |
| uname | Linux dcb5b341b388 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/14/testReport/ |
| Max. process+thread count | 339 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
|
[GitHub] [hadoop] hadoop-yetus commented on pull request #2354: HADOOP-17281 Implement FileSystem.listStatusIterator() in S3AFileSystem
hadoop-yetus commented on pull request #2354:
URL: https://github.com/apache/hadoop/pull/2354#issuecomment-704228306
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 28s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 37s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 23m 42s | | trunk passed |
| +1 :green_heart: | compile | 19m 48s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 19m 34s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 37s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 17s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 32s | | trunk passed |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 18m 53s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 18m 53s | | the patch passed |
| +1 :green_heart: | compile | 17m 4s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 4s | | the patch passed |
| -0 :warning: | checkstyle | 3m 13s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/3/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 2 new + 19 unchanged - 0 fixed = 21 total (was
19) |
| +1 :green_heart: | mvnsite | 2m 25s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 39s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 14s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 3m 44s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 9m 35s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 35s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 179m 55s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2354 |
| Optional Tests | dupname asflicense mvnsite markdownlint compile javac
javadoc mvninstall unit shadedclient findbugs checkstyle |
| uname | Linux 1bb688443c0b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/3/testReport/ |
| Max. process+thread count | 1459 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws
U: . |
| Console output |
[jira] [Work logged] (HADOOP-17281) Implement FileSystem.listStatusIterator() in S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-17281?focusedWorklogId=495871=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495871
]
ASF GitHub Bot logged work on HADOOP-17281:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 12:15
Start Date: 06/Oct/20 12:15
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2354:
URL: https://github.com/apache/hadoop/pull/2354#issuecomment-704228306
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 28s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 37s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 23m 42s | | trunk passed |
| +1 :green_heart: | compile | 19m 48s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 19m 34s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 37s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 17s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 32s | | trunk passed |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 18m 53s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 18m 53s | | the patch passed |
| +1 :green_heart: | compile | 17m 4s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 4s | | the patch passed |
| -0 :warning: | checkstyle | 3m 13s |
[/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/3/artifact/out/diff-checkstyle-root.txt)
| root: The patch generated 2 new + 19 unchanged - 0 fixed = 21 total (was
19) |
| +1 :green_heart: | mvnsite | 2m 25s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 39s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 14s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 3m 44s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 9m 35s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 35s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 179m 55s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2354/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2354 |
| Optional Tests | dupname asflicense mvnsite markdownlint compile javac
javadoc mvninstall unit shadedclient findbugs checkstyle |
| uname | Linux 1bb688443c0b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
[jira] [Commented] (HADOOP-15944) S3AInputStream logging to make it easier to debug file leakage
[ https://issues.apache.org/jira/browse/HADOOP-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208644#comment-17208644 ] Steve Loughran commented on HADOOP-15944: - Also: let's log the full stack trace of any caught EOF when read()/ readFully() catches an EOF and downgrades it to a -1. This will help identify where EOFs are surfacing > S3AInputStream logging to make it easier to debug file leakage > -- > > Key: HADOOP-15944 > URL: https://issues.apache.org/jira/browse/HADOOP-15944 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.1.1 >Reporter: Steve Loughran >Priority: Minor > > Problem: if an app opens too many input streams, then all the http > connections in the S3A pool can be used up; all attempts to do other FS > operations fail timing out for http pool access > Proposed simple solution: log better what's going on with input stream > lifecyce, specifically > # include URL of file in open, reopen & close events > # maybe: Separate logger for these events, though S3A Input stream should be > enough as it doesn't do much else. > # maybe: have some prefix in the events like "Lifecycle", so that you could > use the existing log @ debug, grep for that phrase and look at the printed > URLs to identify what's going on > # stream metrics: expose some of the state of the http connection pool and/or > active input and output streams > Idle output streams don't use up http connections, as they only connect > during block upload. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17296) ABFS: Allow Random Reads to be of Buffer Size
[ https://issues.apache.org/jira/browse/HADOOP-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208604#comment-17208604 ] Mukund Thakur commented on HADOOP-17296: won't setting the fs.azure.readahead.range equal to buffer size in this PR [https://github.com/apache/hadoop/pull/2307] will achieve the same thing? > ABFS: Allow Random Reads to be of Buffer Size > - > > Key: HADOOP-17296 > URL: https://issues.apache.org/jira/browse/HADOOP-17296 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Major > Labels: abfsactive > > ADLS Gen2/ABFS driver is optimized to read only the bytes that are requested > for when the read pattern is random. > It was observed in some spark jobs that though the reads are random, the next > read doesn't skip by a lot and can be served by the earlier read if read was > done in buffer size. As a result the job triggered a higher count of read > calls and resulted in higher job runtime. > When these jobs were run against Gen1 which always reads in buffer size , the > jobs fared well. > In this Jira we try to provide a control over config on random read to be of > requested size or buffer size. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17288) Use shaded guava from thirdparty
[
https://issues.apache.org/jira/browse/HADOOP-17288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208607#comment-17208607
]
Ayush Saxena commented on HADOOP-17288:
---
[~stack] The guava I planned to revert to 11 as of now, that would be used by
the dependencies only, Not by the hadoop code, and people including hadoop
dependencies will have a choice to exclude this {{Gauva-11}} if they themselves
have guava. So, I think it should be safe? The dependencies also doesn't seems
to be very specific with the version, as {{curator}} has already shaded
{{gauva}} apart from 3-4 files, for which we need guava, and those files are
quite stable, there are couple of more dependencies which use guava, they also
doesn't seems to create much issues. All tests passed with {{{gauva-11}} and
{{guava-27}} as well.
So, I prefer staying at {{guava-11}}. Let me know your thoughts?
Plan to address [~vinayakumarb] comments by tomorrow, once you confirm. Thanx..
> Use shaded guava from thirdparty
>
>
> Key: HADOOP-17288
> URL: https://issues.apache.org/jira/browse/HADOOP-17288
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Use the shaded version of guava in hadoop-thirdparty
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2278: HADOOP-17191. ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704137336
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 37s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 59s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 5s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 3s | | trunk passed |
| -0 :warning: | patch | 1m 23s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| -1 :x: | shellcheck | 0m 0s |
[/diff-patch-shellcheck.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/13/artifact/out/diff-patch-shellcheck.txt)
| The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | shelldocs | 0m 13s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 16m 5s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 0m 59s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 79m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle
xml |
| uname | Linux 54956957feae 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495801=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495801
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 09:09
Start Date: 06/Oct/20 09:09
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704137336
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 37s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 59s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 5s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 3s | | trunk passed |
| -0 :warning: | patch | 1m 23s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| -1 :x: | shellcheck | 0m 0s |
[/diff-patch-shellcheck.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/13/artifact/out/diff-patch-shellcheck.txt)
| The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | shelldocs | 0m 13s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 16m 5s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 0m 59s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 79m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle
xml |
| uname | Linux 54956957feae 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[GitHub] [hadoop] hadoop-yetus commented on pull request #2297: HADOOP-17125. Using snappy-java in SnappyCodec
hadoop-yetus commented on pull request #2297:
URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704136004
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 36s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 24m 2s | | trunk passed |
| +1 :green_heart: | compile | 19m 49s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 17m 5s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 20m 56s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 21s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 6m 29s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 7m 8s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 45s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +0 :ok: | findbugs | 0m 25s | | branch/hadoop-project no findbugs
output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 23s | | branch/hadoop-project-dist no
findbugs output file (findbugsXml.xml) |
| -0 :warning: | patch | 1m 7s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 35s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 21m 15s | | the patch passed |
| +1 :green_heart: | compile | 19m 18s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| -1 :x: | cc | 19m 18s |
[/diff-compile-cc-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2297/22/artifact/out/diff-compile-cc-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
| root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 40 new + 123 unchanged -
40 fixed = 163 total (was 163) |
| +1 :green_heart: | golang | 19m 18s | | the patch passed |
| +1 :green_heart: | javac | 19m 18s | | the patch passed |
| +1 :green_heart: | compile | 17m 9s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| -1 :x: | cc | 17m 9s |
[/diff-compile-cc-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2297/22/artifact/out/diff-compile-cc-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt)
| root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 36 new + 127
unchanged - 36 fixed = 163 total (was 163) |
| +1 :green_heart: | golang | 17m 9s | | the patch passed |
| +1 :green_heart: | javac | 17m 9s | | the patch passed |
| +1 :green_heart: | checkstyle | 2m 50s | | root: The patch generated
0 new + 140 unchanged - 3 fixed = 140 total (was 143) |
| +1 :green_heart: | mvnsite | 17m 36s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 18s | | There were no new
shelldocs issues. |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 5s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 11s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 6m 25s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 7m 9s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | findbugs | 0m 23s | | hadoop-project has no data from
findbugs |
| +0 :ok: | findbugs | 0m 24s | | hadoop-project-dist has no data from
findbugs |
_ Other Tests _
[jira] [Work logged] (HADOOP-17125) Using snappy-java in SnappyCodec
[
https://issues.apache.org/jira/browse/HADOOP-17125?focusedWorklogId=495800=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495800
]
ASF GitHub Bot logged work on HADOOP-17125:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 09:06
Start Date: 06/Oct/20 09:06
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2297:
URL: https://github.com/apache/hadoop/pull/2297#issuecomment-704136004
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 5m 36s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 24m 2s | | trunk passed |
| +1 :green_heart: | compile | 19m 49s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 17m 5s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 20m 56s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 21s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 6m 29s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 7m 8s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 45s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +0 :ok: | findbugs | 0m 25s | | branch/hadoop-project no findbugs
output file (findbugsXml.xml) |
| +0 :ok: | findbugs | 0m 23s | | branch/hadoop-project-dist no
findbugs output file (findbugsXml.xml) |
| -0 :warning: | patch | 1m 7s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 35s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 21m 15s | | the patch passed |
| +1 :green_heart: | compile | 19m 18s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| -1 :x: | cc | 19m 18s |
[/diff-compile-cc-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2297/22/artifact/out/diff-compile-cc-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
| root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 40 new + 123 unchanged -
40 fixed = 163 total (was 163) |
| +1 :green_heart: | golang | 19m 18s | | the patch passed |
| +1 :green_heart: | javac | 19m 18s | | the patch passed |
| +1 :green_heart: | compile | 17m 9s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| -1 :x: | cc | 17m 9s |
[/diff-compile-cc-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2297/22/artifact/out/diff-compile-cc-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt)
| root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 36 new + 127
unchanged - 36 fixed = 163 total (was 163) |
| +1 :green_heart: | golang | 17m 9s | | the patch passed |
| +1 :green_heart: | javac | 17m 9s | | the patch passed |
| +1 :green_heart: | checkstyle | 2m 50s | | root: The patch generated
0 new + 140 unchanged - 3 fixed = 140 total (was 143) |
| +1 :green_heart: | mvnsite | 17m 36s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | There were no new
shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 18s | | There were no new
shelldocs issues. |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 5s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 11s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc |
[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
[
https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=495799=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495799
]
ASF GitHub Bot logged work on HADOOP-17191:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 09:05
Start Date: 06/Oct/20 09:05
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704135023
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 5s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 19s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 42s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 1s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 59s | | trunk passed |
| -0 :warning: | patch | 1m 17s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| -1 :x: | shellcheck | 0m 2s |
[/diff-patch-shellcheck.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/12/artifact/out/diff-patch-shellcheck.txt)
| The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 59s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 1s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 79m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle
xml |
| uname | Linux 11d2b02cf7bb 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[GitHub] [hadoop] hadoop-yetus commented on pull request #2278: HADOOP-17191. ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results
hadoop-yetus commented on pull request #2278:
URL: https://github.com/apache/hadoop/pull/2278#issuecomment-704135023
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 5s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 19s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 42s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 1s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 59s | | trunk passed |
| -0 :warning: | patch | 1m 17s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| -1 :x: | shellcheck | 0m 2s |
[/diff-patch-shellcheck.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/12/artifact/out/diff-patch-shellcheck.txt)
| The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | shelldocs | 0m 14s | | The patch generated 0 new
+ 104 unchanged - 132 fixed = 104 total (was 236) |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 59s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 1s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 79m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2278/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2278 |
| Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs
markdownlint compile javac javadoc mvninstall shadedclient findbugs checkstyle
xml |
| uname | Linux 11d2b02cf7bb 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2342: HADOOP-17288. Use shaded guava from thirdparty.
vinayakumarb commented on a change in pull request #2342:
URL: https://github.com/apache/hadoop/pull/2342#discussion_r500116361
##
File path: hadoop-common-project/hadoop-auth/pom.xml
##
@@ -234,6 +235,24 @@
${basedir}/dev-support/findbugsExcludeFile.xml
+
+com.google.code.maven-replacer-plugin
+replacer
+
+
+replace-sources
+
+ false
+
+
+
+replace-test-sources
+
+ false
+
+
+
+
Review comment:
Instead of replacing guava references along with protobuf references,
create a separate replacer-execution in hadoop-project/pom.xml and enable by
default for all projects.
```
replace-guava
process-sources
replace
false
${basedir}
src/main/java/**/*.java
src/test/java/**/*.java
([^\.])com.google.common
$1${hadoop-thirdparty-shaded-guava-prefix}
```
2. Exclude guava-replacement in hadoop-yarn-csi module. yarn-csi seems to
have separate classpath, and it explicitly uses guava-20. This will not affect
any downstreams.
3. Replaces all dependencies on guava to hadoop-shaded-guava (except
yarn-csi)
4. In hadoop-common/pom.xml, Keep both guava (11.0.2) and
hadoop-shaded-guava as dependencies. Of course code should use shaded version
of guava. version 11.0.2 is used in curator libs during runtime.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17288) Use shaded guava from thirdparty
[
https://issues.apache.org/jira/browse/HADOOP-17288?focusedWorklogId=495796=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495796
]
ASF GitHub Bot logged work on HADOOP-17288:
---
Author: ASF GitHub Bot
Created on: 06/Oct/20 08:58
Start Date: 06/Oct/20 08:58
Worklog Time Spent: 10m
Work Description: vinayakumarb commented on a change in pull request
#2342:
URL: https://github.com/apache/hadoop/pull/2342#discussion_r500116361
##
File path: hadoop-common-project/hadoop-auth/pom.xml
##
@@ -234,6 +235,24 @@
${basedir}/dev-support/findbugsExcludeFile.xml
+
+com.google.code.maven-replacer-plugin
+replacer
+
+
+replace-sources
+
+ false
+
+
+
+replace-test-sources
+
+ false
+
+
+
+
Review comment:
Instead of replacing guava references along with protobuf references,
create a separate replacer-execution in hadoop-project/pom.xml and enable by
default for all projects.
```
replace-guava
process-sources
replace
false
${basedir}
src/main/java/**/*.java
src/test/java/**/*.java
([^\.])com.google.common
$1${hadoop-thirdparty-shaded-guava-prefix}
```
2. Exclude guava-replacement in hadoop-yarn-csi module. yarn-csi seems to
have separate classpath, and it explicitly uses guava-20. This will not affect
any downstreams.
3. Replaces all dependencies on guava to hadoop-shaded-guava (except
yarn-csi)
4. In hadoop-common/pom.xml, Keep both guava (11.0.2) and
hadoop-shaded-guava as dependencies. Of course code should use shaded version
of guava. version 11.0.2 is used in curator libs during runtime.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 495796)
Time Spent: 0.5h (was: 20m)
> Use shaded guava from thirdparty
>
>
> Key: HADOOP-17288
> URL: https://issues.apache.org/jira/browse/HADOOP-17288
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Use the shaded version of guava in hadoop-thirdparty
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2360: HDFS-15613. RBF: Router FSCK fails after HDFS-14442.
hadoop-yetus commented on pull request #2360:
URL: https://github.com/apache/hadoop/pull/2360#issuecomment-704097550
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 24s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 42s | | trunk passed |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 49s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 34s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 49s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 44s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 42s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 41s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 17m 15s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 52s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 13m 18s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 105m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2360/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2360 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 183a5e9dbcd5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4347a5c9556 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2360/5/testReport/ |
| Max. process+thread count | 3068 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2360/5/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (HADOOP-17300) FileSystem.DirListingIterator.next() call should return NoSuchElementException
Mukund Thakur created HADOOP-17300: -- Summary: FileSystem.DirListingIterator.next() call should return NoSuchElementException Key: HADOOP-17300 URL: https://issues.apache.org/jira/browse/HADOOP-17300 Project: Hadoop Common Issue Type: Bug Components: common, fs Reporter: Mukund Thakur Assignee: Mukund Thakur FileSystem.DirListingIterator.next() call should return NoSuchElementException rather than IllegalStateException CC [~ste...@apache.org] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2360: HDFS-15613. RBF: Router FSCK fails after HDFS-14442.
aajisaka commented on pull request #2360: URL: https://github.com/apache/hadoop/pull/2360#issuecomment-704061579 > hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload The test failure is not related and the test passed in my local environment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
