Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2031089945
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java:
##
@@ -438,6 +438,11 @@ final void parseResponse(final byte[] buffer,
method, getMaskedUrl(), ex.getMessage());
log.debug("IO Error: ", ex);
throw ex;
+ } catch (Exception ex) {
+log.warn("Unexpected error: {} {}: {}",
Review Comment:
We were doing same for other type of exceptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2031087291
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java:
##
@@ -500,6 +505,20 @@ private void parseBlockListResponse(final InputStream
stream) throws IOException
blockIdList = client.parseBlockListResponse(stream);
}
+ private void parseListPathResponse(final InputStream stream) throws
IOException {
Review Comment:
Taken
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java:
##
@@ -500,6 +505,20 @@ private void parseBlockListResponse(final InputStream
stream) throws IOException
blockIdList = client.parseBlockListResponse(stream);
}
+ private void parseListPathResponse(final InputStream stream) throws
IOException {
+if (stream == null || blockIdList != null) {
Review Comment:
Was buggy, fixed it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anmolanmol1234 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2031109387
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemListStatus.java:
##
@@ -176,6 +183,31 @@ public void testListPathTracingContext() throws Exception {
Mockito.verify(spiedTracingContext, times(0)).constructHeader(any(),
any(), any());
}
+ @Test
+ public void testListPathParsingFailure() throws Exception {
+assumeBlobServiceType();
+AzureBlobFileSystem spiedFs = Mockito.spy(getFileSystem());
+AzureBlobFileSystemStore spiedStore = Mockito.spy(spiedFs.getAbfsStore());
+AbfsBlobClient spiedClient = Mockito.spy(spiedStore.getClientHandler()
+.getBlobClient());
+Mockito.doReturn(spiedStore).when(spiedFs).getAbfsStore();
+Mockito.doReturn(spiedClient).when(spiedStore).getClient();
+
+Mockito.doThrow(new
SocketException(CONNECTION_RESET_MESSAGE)).when(spiedClient).filterDuplicateEntriesAndRenamePendingFiles(any(),
any());
Review Comment:
Can we reduce the retry count and verify that request was retried ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2782217406
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 45s | | trunk passed |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 5s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 22s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 15s | | the patch passed |
| +1 :green_heart: | shadedclient | 39m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 53s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 134m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7582 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 87178c95bb6d 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10
21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 37f01918852faa2e6bda63c8c1dd66a22d46c5c7 |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/3/testReport/ |
| Max. process+thread count | 533 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 merged PR #7585: URL: https://github.com/apache/hadoop/pull/7585 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on PR #7585: URL: https://github.com/apache/hadoop/pull/7585#issuecomment-2791486944 HNS-OAuth-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 3 [WARNING] Tests run: 809, Failures: 0, Errors: 0, Skipped: 172 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 30 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 23 HNS-SharedKey-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 4 [WARNING] Tests run: 809, Failures: 0, Errors: 0, Skipped: 123 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 30 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 10 NonHNS-SharedKey-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 793, Failures: 0, Errors: 0, Skipped: 366 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 31 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 11 AppendBlob-HNS-OAuth-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 3 [WARNING] Tests run: 809, Failures: 0, Errors: 0, Skipped: 179 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 54 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 23 NonHNS-SharedKey-Blob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 793, Failures: 0, Errors: 0, Skipped: 292 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 27 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 11 NonHNS-OAuth-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 793, Failures: 0, Errors: 0, Skipped: 370 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 31 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24 NonHNS-OAuth-Blob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 793, Failures: 0, Errors: 0, Skipped: 296 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 27 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24 AppendBlob-NonHNS-OAuth-Blob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 793, Failures: 0, Errors: 0, Skipped: 316 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 51 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24 HNS-Oauth-DFS-IngressBlob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 3 [WARNING] Tests run: 809, Failures: 0, Errors: 0, Skipped: 301 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 30 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 23 NonHNS-OAuth-DFS-IngressBlob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 793, Failures: 0, Errors: 0, Skipped: 368 [WARNING] Tests run: 180, Failures: 0, Errors: 0, Skipped: 31 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7585:
URL: https://github.com/apache/hadoop/pull/7585#issuecomment-2786294933
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 11m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ branch-3.4 Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 53s | | branch-3.4 passed |
| +1 :green_heart: | compile | 0m 40s | | branch-3.4 passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 37s | | branch-3.4 passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 34s | | branch-3.4 passed |
| +1 :green_heart: | mvnsite | 0m 41s | | branch-3.4 passed |
| +1 :green_heart: | javadoc | 0m 40s | | branch-3.4 passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 36s | | branch-3.4 passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 8s | | branch-3.4 passed |
| +1 :green_heart: | shadedclient | 32m 16s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 32m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 33s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 124m 13s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7585/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7585 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 86863d85e990 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18
17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.4 / 5b97b28bdbe0f8b9b5a2d7085ca210c0b667d5d4 |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7585/1/testReport/ |
| Max. process+thread count | 552 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7585/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 merged PR #7582: URL: https://github.com/apache/hadoop/pull/7582 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2785273594
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 48s | | trunk passed |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 33s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 43s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 133m 31s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7582 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux cc84c03266e5 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10
21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fde17834bcfe03986216a29655cd340e24ad5b10 |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/9/testReport/ |
| Max. process+thread count | 587 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/9/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on PR #7582: URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2785127499 HNS-OAuth-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 3 [WARNING] Tests run: 798, Failures: 0, Errors: 0, Skipped: 161 [WARNING] Tests run: 155, Failures: 0, Errors: 0, Skipped: 5 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 23 HNS-SharedKey-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 4 [WARNING] Tests run: 801, Failures: 0, Errors: 0, Skipped: 114 [WARNING] Tests run: 155, Failures: 0, Errors: 0, Skipped: 5 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 10 NonHNS-SharedKey-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 640, Failures: 0, Errors: 0, Skipped: 212 [WARNING] Tests run: 155, Failures: 0, Errors: 0, Skipped: 6 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 11 AppendBlob-HNS-OAuth-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 3 [WARNING] Tests run: 798, Failures: 0, Errors: 0, Skipped: 168 [WARNING] Tests run: 123, Failures: 0, Errors: 0, Skipped: 6 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 23 NonHNS-SharedKey-Blob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 643, Failures: 0, Errors: 0, Skipped: 141 [WARNING] Tests run: 146, Failures: 0, Errors: 0, Skipped: 3 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 11 NonHNS-OAuth-DFS [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 637, Failures: 0, Errors: 0, Skipped: 214 [WARNING] Tests run: 155, Failures: 0, Errors: 0, Skipped: 6 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24 NonHNS-OAuth-Blob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 640, Failures: 0, Errors: 0, Skipped: 143 [WARNING] Tests run: 155, Failures: 0, Errors: 0, Skipped: 3 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24 AppendBlob-NonHNS-OAuth-Blob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 638, Failures: 0, Errors: 0, Skipped: 161 [WARNING] Tests run: 132, Failures: 0, Errors: 0, Skipped: 4 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24 HNS-Oauth-DFS-IngressBlob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 3 [WARNING] Tests run: 672, Failures: 0, Errors: 0, Skipped: 164 [WARNING] Tests run: 155, Failures: 0, Errors: 0, Skipped: 5 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 23 NonHNS-OAuth-DFS-IngressBlob [WARNING] Tests run: 174, Failures: 0, Errors: 0, Skipped: 10 [WARNING] Tests run: 637, Failures: 0, Errors: 0, Skipped: 212 [WARNING] Tests run: 146, Failures: 0, Errors: 0, Skipped: 6 [WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2783990201
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 54s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 36m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 1m 12s |
[/new-spotbugs-hadoop-tools_hadoop-azure.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/8/artifact/out/new-spotbugs-hadoop-tools_hadoop-azure.html)
| hadoop-tools/hadoop-azure generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 38m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 49s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 127m 9s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-tools/hadoop-azure |
| |
org.apache.hadoop.fs.azurebfs.services.AbfsHttpOperation.getListResultData()
may expose internal representation by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:[line 226] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7582 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 1199960dd5fa 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10
21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 5b9dfa5ff9977cd1be60a332ff34ba60f25d123c |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/8/testReport/ |
| Max. process+thread count | 737 (vs. ulimit of 5500) |
| modules | C
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2783727608
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 18s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 41m 6s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 41s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 23s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| -1 :x: | javadoc | 0m 38s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/7/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-tools_hadoop-azure-jdkUbuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 generated 1 new + 10
unchanged - 0 fixed = 11 total (was 10) |
| -1 :x: | javadoc | 0m 30s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/7/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06.txt)
|
hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06
with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 generated 1
new + 10 unchanged - 0 fixed = 11 total (was 10) |
| -1 :x: | spotbugs | 1m 22s |
[/new-spotbugs-hadoop-tools_hadoop-azure.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/7/artifact/out/new-spotbugs-hadoop-tools_hadoop-azure.html)
| hadoop-tools/hadoop-azure generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 39m 59s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 43s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 140m 7s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-tools/hadoop-azure |
| |
org.apache.hadoop.fs.azurebfs.services.AbfsHttpOperation.getListResultData()
may expose internal representation by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:[line 226] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7582 |
| Optional Tests | dupname asflicense compile java
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2783487717
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 27s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 12s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 21s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/6/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 3 new + 1 unchanged - 0
fixed = 4 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| -1 :x: | javadoc | 0m 30s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/6/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04.txt)
| hadoop-tools_hadoop-azure-jdkUbuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 generated 1 new + 10
unchanged - 0 fixed = 11 total (was 10) |
| -1 :x: | javadoc | 0m 25s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/6/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06.txt)
|
hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06
with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 generated 1
new + 10 unchanged - 0 fixed = 11 total (was 10) |
| -1 :x: | spotbugs | 1m 10s |
[/new-spotbugs-hadoop-tools_hadoop-azure.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/6/artifact/out/new-spotbugs-hadoop-tools_hadoop-azure.html)
| hadoop-tools/hadoop-azure generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 39m 23s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 44s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 133m 42s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-tools/hadoop-azure |
| |
org.apache.hadoop.fs.azurebfs.services.AbfsHttpOperation.getListResultData()
may expose internal representation by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:[line 226] |
| Subsystem | Report/Notes |
|--:|:-|
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2031189514
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemListStatus.java:
##
@@ -176,6 +183,31 @@ public void testListPathTracingContext() throws Exception {
Mockito.verify(spiedTracingContext, times(0)).constructHeader(any(),
any(), any());
}
+ @Test
+ public void testListPathParsingFailure() throws Exception {
+assumeBlobServiceType();
+AzureBlobFileSystem spiedFs = Mockito.spy(getFileSystem());
+AzureBlobFileSystemStore spiedStore = Mockito.spy(spiedFs.getAbfsStore());
+AbfsBlobClient spiedClient = Mockito.spy(spiedStore.getClientHandler()
+.getBlobClient());
+Mockito.doReturn(spiedStore).when(spiedFs).getAbfsStore();
+Mockito.doReturn(spiedClient).when(spiedStore).getClient();
+
+Mockito.doThrow(new
SocketException(CONNECTION_RESET_MESSAGE)).when(spiedClient).filterDuplicateEntriesAndRenamePendingFiles(any(),
any());
Review Comment:
This particular test is to make sure retry does not happen as here exception
is injected in parsing list result from local stream.
I have modified the test `testListPathTracingContext` to assert that
exception thrown within httpOperation.processResponse() is retried.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
manika137 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2030758782
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestApacheClientConnectionPool.java:
##
@@ -67,6 +68,7 @@ public ITestApacheClientConnectionPool() throws Exception {
public void testKacIsClosed() throws Throwable {
Configuration configuration = new Configuration(getRawConfiguration());
configuration.set(FS_AZURE_NETWORKING_LIBRARY, APACHE_HTTP_CLIENT.name());
+configuration.unset(FS_AZURE_METRIC_FORMAT);
Review Comment:
why do we need to unset this config here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
bhattmanish98 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2030748158
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##
@@ -1606,10 +1607,7 @@ public Hashtable
getXMSProperties(AbfsHttpOperation result)
public ListResponseData parseListPathResults(AbfsHttpOperation result, URI
uri)
throws AzureBlobFileSystemException {
BlobListResultSchema listResultSchema;
-try (InputStream stream = result.getListResultStream()) {
- if (stream == null) {
-return null;
- }
+try (InputStream stream = new
ByteArrayInputStream(result.getListResultData())) {
Review Comment:
Will there be any issue here in case result.getListResultData() is null?
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java:
##
@@ -438,6 +438,11 @@ final void parseResponse(final byte[] buffer,
method, getMaskedUrl(), ex.getMessage());
log.debug("IO Error: ", ex);
throw ex;
+ } catch (Exception ex) {
+log.warn("Unexpected error: {} {}: {}",
Review Comment:
Why do we need both log.warn and log.debug here?
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##
@@ -1620,19 +1618,17 @@ public ListResponseData
parseListPathResults(AbfsHttpOperation result, URI uri)
LOG.debug("ListBlobs listed {} blobs with {} as continuation token",
listResultSchema.paths().size(),
listResultSchema.getNextMarker());
- } catch (SAXException | IOException e) {
-throw new AbfsDriverException(e);
+return filterDuplicateEntriesAndRenamePendingFiles(listResultSchema,
uri);
+ } catch (SAXException | IOException ex) {
+throw new AbfsDriverException(ERR_BLOB_LIST_PARSING, ex);
Review Comment:
NIT: We can add URI in the error message for better visibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2031086890
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsDfsClient.java:
##
@@ -1501,9 +1504,13 @@ public ListResponseData
parseListPathResults(AbfsHttpOperation result, URI uri)
listResponseData.setContinuationToken(
getContinuationFromResponse(result));
return listResponseData;
+} catch (AbfsDriverException ex) {
+ // Throw as it is to avoid multiple wrapping.
+ LOG.error("Unable to deserialize list results for Uri {}", uri != null ?
uri.toString(): "NULL", ex);
+ throw ex;
} catch (IOException ex) {
- LOG.error("Unable to deserialize list results for Uri {}",
uri.toString(), ex);
- throw new AbfsDriverException(ERR_DFS_LIST_PARSING, ex);
+ LOG.error("Unable to deserialize list results for Uri {}", uri != null ?
uri.toString(): "NULL", ex);
+ throw new AbfsDriverException(ex);
Review Comment:
Taken
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2031088962
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##
@@ -1606,10 +1607,7 @@ public Hashtable
getXMSProperties(AbfsHttpOperation result)
public ListResponseData parseListPathResults(AbfsHttpOperation result, URI
uri)
throws AzureBlobFileSystemException {
BlobListResultSchema listResultSchema;
-try (InputStream stream = result.getListResultStream()) {
- if (stream == null) {
-return null;
- }
+try (InputStream stream = new
ByteArrayInputStream(result.getListResultData())) {
Review Comment:
Yes, it can lead to NPE. Have modified to catch all the exceptions and wrap
it around AbfsDriverException and throw back to user
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2031090896
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestApacheClientConnectionPool.java:
##
@@ -67,6 +68,7 @@ public ITestApacheClientConnectionPool() throws Exception {
public void testKacIsClosed() throws Throwable {
Configuration configuration = new Configuration(getRawConfiguration());
configuration.set(FS_AZURE_NETWORKING_LIBRARY, APACHE_HTTP_CLIENT.name());
+configuration.unset(FS_AZURE_METRIC_FORMAT);
Review Comment:
This test was failing when this config was present in test xml files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2031087943
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##
@@ -1620,19 +1618,17 @@ public ListResponseData
parseListPathResults(AbfsHttpOperation result, URI uri)
LOG.debug("ListBlobs listed {} blobs with {} as continuation token",
listResultSchema.paths().size(),
listResultSchema.getNextMarker());
- } catch (SAXException | IOException e) {
-throw new AbfsDriverException(e);
+return filterDuplicateEntriesAndRenamePendingFiles(listResultSchema,
uri);
+ } catch (SAXException | IOException ex) {
+throw new AbfsDriverException(ERR_BLOB_LIST_PARSING, ex);
Review Comment:
URI is already in log messages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2783078898
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 9s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 41m 28s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 1m 8s |
[/new-spotbugs-hadoop-tools_hadoop-azure.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/5/artifact/out/new-spotbugs-hadoop-tools_hadoop-azure.html)
| hadoop-tools/hadoop-azure generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 38m 48s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 43s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 135m 22s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-tools/hadoop-azure |
| |
org.apache.hadoop.fs.azurebfs.services.AbfsHttpOperation.getListResultData()
may expose internal representation by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:[line 226] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7582 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d0a63866147f 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10
21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 60f1c38ac0d8bb2f70cdee0e53fbd6c25b912389 |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/5/testReport/ |
| Max. process+thread count | 536 (vs. ulimit of 5500) |
| modules | C
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anujmodi2021 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2031074033
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##
@@ -1620,19 +1618,17 @@ public ListResponseData
parseListPathResults(AbfsHttpOperation result, URI uri)
LOG.debug("ListBlobs listed {} blobs with {} as continuation token",
listResultSchema.paths().size(),
listResultSchema.getNextMarker());
- } catch (SAXException | IOException e) {
-throw new AbfsDriverException(e);
+return filterDuplicateEntriesAndRenamePendingFiles(listResultSchema,
uri);
+ } catch (SAXException | IOException ex) {
+throw new AbfsDriverException(ERR_BLOB_LIST_PARSING, ex);
}
-} catch (IOException e) {
- LOG.error("Unable to deserialize list results for uri {}",
uri.toString(), e);
- throw new AbfsDriverException(ERR_BLOB_LIST_PARSING, e);
-}
-
-try {
- return filterDuplicateEntriesAndRenamePendingFiles(listResultSchema,
uri);
-} catch (IOException e) {
- LOG.error("Unable to filter list results for uri {}", uri.toString(), e);
- throw new AbfsDriverException(ERR_BLOB_LIST_PARSING, e);
+} catch (AbfsDriverException ex) {
+ // Throw as it is to avoid multiple wrapping.
+ LOG.error("Unable to deserialize list results for Uri {}", uri != null ?
uri.toString(): "NULL", ex);
Review Comment:
uri can be null for the cases where this API is called by internal
operations like in create flow.
If list call is coming from user, uri won't be null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anmolanmol1234 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2030570194
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java:
##
@@ -500,6 +505,20 @@ private void parseBlockListResponse(final InputStream
stream) throws IOException
blockIdList = client.parseBlockListResponse(stream);
}
+ private void parseListPathResponse(final InputStream stream) throws
IOException {
+if (stream == null || blockIdList != null) {
Review Comment:
how is blockIdList check relevant here ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anmolanmol1234 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2030569148
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java:
##
@@ -500,6 +505,20 @@ private void parseBlockListResponse(final InputStream
stream) throws IOException
blockIdList = client.parseBlockListResponse(stream);
}
+ private void parseListPathResponse(final InputStream stream) throws
IOException {
Review Comment:
add javadocs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2782217572
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 3s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 1s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 16s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 22s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 13s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 43s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 51s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 136m 20s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7582 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e6f4c2dbe861 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10
21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 37f01918852faa2e6bda63c8c1dd66a22d46c5c7 |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/2/testReport/ |
| Max. process+thread count | 578 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2782214036
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 8m 19s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 25m 18s | | trunk passed |
| +1 :green_heart: | compile | 0m 24s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 21s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 19s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 0m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 16s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 16s | | the patch passed |
| +1 :green_heart: | compile | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 18s | | the patch passed |
| +1 :green_heart: | compile | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| -1 :x: | spotbugs | 0m 42s |
[/new-spotbugs-hadoop-tools_hadoop-azure.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/4/artifact/out/new-spotbugs-hadoop-tools_hadoop-azure.html)
| hadoop-tools/hadoop-azure generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 21m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 21s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 24s | | The patch does not
generate ASF License warnings. |
| | | 85m 14s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-tools/hadoop-azure |
| |
org.apache.hadoop.fs.azurebfs.services.AbfsHttpOperation.getListResultData()
may expose internal representation by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:by returning
AbfsHttpOperation.listResultData At AbfsHttpOperation.java:[line 226] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7582 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux fe256f5f7182 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18
17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2689733f590535af4a627e7018b6dc8d7c53ea1a |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/4/testReport/ |
| Max. process+thread count | 547 (vs. ulimit of 5500) |
| modules | C
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anmolanmol1234 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2030563300
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##
@@ -1620,19 +1618,17 @@ public ListResponseData
parseListPathResults(AbfsHttpOperation result, URI uri)
LOG.debug("ListBlobs listed {} blobs with {} as continuation token",
listResultSchema.paths().size(),
listResultSchema.getNextMarker());
- } catch (SAXException | IOException e) {
-throw new AbfsDriverException(e);
+return filterDuplicateEntriesAndRenamePendingFiles(listResultSchema,
uri);
+ } catch (SAXException | IOException ex) {
+throw new AbfsDriverException(ERR_BLOB_LIST_PARSING, ex);
}
-} catch (IOException e) {
- LOG.error("Unable to deserialize list results for uri {}",
uri.toString(), e);
- throw new AbfsDriverException(ERR_BLOB_LIST_PARSING, e);
-}
-
-try {
- return filterDuplicateEntriesAndRenamePendingFiles(listResultSchema,
uri);
-} catch (IOException e) {
- LOG.error("Unable to filter list results for uri {}", uri.toString(), e);
- throw new AbfsDriverException(ERR_BLOB_LIST_PARSING, e);
+} catch (AbfsDriverException ex) {
+ // Throw as it is to avoid multiple wrapping.
+ LOG.error("Unable to deserialize list results for Uri {}", uri != null ?
uri.toString(): "NULL", ex);
Review Comment:
uri should not be null if we are reaching till this part of the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
anmolanmol1234 commented on code in PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#discussion_r2030561973
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsDfsClient.java:
##
@@ -1501,9 +1504,13 @@ public ListResponseData
parseListPathResults(AbfsHttpOperation result, URI uri)
listResponseData.setContinuationToken(
getContinuationFromResponse(result));
return listResponseData;
+} catch (AbfsDriverException ex) {
+ // Throw as it is to avoid multiple wrapping.
+ LOG.error("Unable to deserialize list results for Uri {}", uri != null ?
uri.toString(): "NULL", ex);
+ throw ex;
} catch (IOException ex) {
- LOG.error("Unable to deserialize list results for Uri {}",
uri.toString(), ex);
- throw new AbfsDriverException(ERR_DFS_LIST_PARSING, ex);
+ LOG.error("Unable to deserialize list results for Uri {}", uri != null ?
uri.toString(): "NULL", ex);
+ throw new AbfsDriverException(ex);
Review Comment:
throw new AbfsDriverException(ERR_DFS_LIST_PARSING, ex); add the message
here as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19531. [ABFS][FnsOverBlob] Streaming List Path Result Should Happen Inside Retry Loop [hadoop]
hadoop-yetus commented on PR #7582:
URL: https://github.com/apache/hadoop/pull/7582#issuecomment-2780668739
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 21m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 40m 25s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/1/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 33s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 43s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 153m 48s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7582 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d4259183eced 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10
21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e3f272aaa285701ddea68e9fca0364940811e166 |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/1/testReport/ |
| Max. process+thread count | 588 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7582/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is
