[jira] [Commented] (HDFS-15900) RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode
[
https://issues.apache.org/jira/browse/HDFS-15900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314670#comment-17314670
]
Harunobu Daikoku commented on HDFS-15900:
-
Made a PR for branch-3.2: https://github.com/apache/hadoop/pull/2866
> RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode
> ---
>
> Key: HDFS-15900
> URL: https://issues.apache.org/jira/browse/HDFS-15900
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Affects Versions: 3.3.0
>Reporter: Harunobu Daikoku
>Assignee: Harunobu Daikoku
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0
>
> Attachments: image.png
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> We observed that when a NameNode becomes UNAVAILABLE, the corresponding
> blockpool id in MembershipStoreImpl#activeNamespaces on dfsrouter
> unintentionally sets to empty, its initial value.
> !image.png|height=250!
> As a result of this, concat operations through dfsrouter fail with the
> following error as it cannot resolve the block id in the recognized active
> namespaces.
> {noformat}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RemoteException):
> Cannot locate a nameservice for block pool BP-...
> {noformat}
> A possible fix is to ignore UNAVAILABLE NameNode registrations, and set
> proper namespace information obtained from available NameNode registrations
> when constructing the cache of active namespaces.
>
> [https://github.com/apache/hadoop/blob/rel/release-3.3.0/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/impl/MembershipStoreImpl.java#L207-L221]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15900) RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode
[
https://issues.apache.org/jira/browse/HDFS-15900?focusedWorklogId=576698=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576698
]
ASF GitHub Bot logged work on HDFS-15900:
-
Author: ASF GitHub Bot
Created on: 05/Apr/21 05:09
Start Date: 05/Apr/21 05:09
Worklog Time Spent: 10m
Work Description: hdaikoku commented on a change in pull request #2866:
URL: https://github.com/apache/hadoop/pull/2866#discussion_r606920944
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRPCClientRetries.java
##
@@ -155,10 +155,10 @@ public void testRetryWhenOneNameServiceDown() throws
Exception {
// Renew lease for the DFS client, it will succeed.
routerProtocol.renewLease(client.getClientName());
-// Verify the retry times, it will retry one time for ns0.
+// Verify the retry times, it should succeed with no retry as long as at
least one of the nameservices is ACTIVE.
FederationRPCMetrics rpcMetrics = routerContext.getRouter()
.getRpcServer().getRPCMetrics();
-assertEquals(1, rpcMetrics.getProxyOpRetries());
+assertEquals(0, rpcMetrics.getProxyOpRetries());
Review comment:
With this patch, ns0 is no longer a target nameservice of the renewLease
RPC and thus there should be no retries.
https://github.com/apache/hadoop/blob/02d578a9a1d2e084b609c24640c51a783a598464/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java#L595-L596
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 576698)
Time Spent: 4.5h (was: 4h 20m)
> RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode
> ---
>
> Key: HDFS-15900
> URL: https://issues.apache.org/jira/browse/HDFS-15900
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Affects Versions: 3.3.0
>Reporter: Harunobu Daikoku
>Assignee: Harunobu Daikoku
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0
>
> Attachments: image.png
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> We observed that when a NameNode becomes UNAVAILABLE, the corresponding
> blockpool id in MembershipStoreImpl#activeNamespaces on dfsrouter
> unintentionally sets to empty, its initial value.
> !image.png|height=250!
> As a result of this, concat operations through dfsrouter fail with the
> following error as it cannot resolve the block id in the recognized active
> namespaces.
> {noformat}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RemoteException):
> Cannot locate a nameservice for block pool BP-...
> {noformat}
> A possible fix is to ignore UNAVAILABLE NameNode registrations, and set
> proper namespace information obtained from available NameNode registrations
> when constructing the cache of active namespaces.
>
> [https://github.com/apache/hadoop/blob/rel/release-3.3.0/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/impl/MembershipStoreImpl.java#L207-L221]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15900) RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode
[
https://issues.apache.org/jira/browse/HDFS-15900?focusedWorklogId=576697=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576697
]
ASF GitHub Bot logged work on HDFS-15900:
-
Author: ASF GitHub Bot
Created on: 05/Apr/21 05:08
Start Date: 05/Apr/21 05:08
Worklog Time Spent: 10m
Work Description: hdaikoku commented on a change in pull request #2866:
URL: https://github.com/apache/hadoop/pull/2866#discussion_r606920944
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRPCClientRetries.java
##
@@ -155,10 +155,10 @@ public void testRetryWhenOneNameServiceDown() throws
Exception {
// Renew lease for the DFS client, it will succeed.
routerProtocol.renewLease(client.getClientName());
-// Verify the retry times, it will retry one time for ns0.
+// Verify the retry times, it should succeed with no retry as long as at
least one of the nameservices is ACTIVE.
FederationRPCMetrics rpcMetrics = routerContext.getRouter()
.getRpcServer().getRPCMetrics();
-assertEquals(1, rpcMetrics.getProxyOpRetries());
+assertEquals(0, rpcMetrics.getProxyOpRetries());
Review comment:
With this patch, ns0 is no longer a target namespace of the renewLease
RPC and thus there should be no retries.
https://github.com/apache/hadoop/blob/02d578a9a1d2e084b609c24640c51a783a598464/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java#L595-L596
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 576697)
Time Spent: 4h 20m (was: 4h 10m)
> RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode
> ---
>
> Key: HDFS-15900
> URL: https://issues.apache.org/jira/browse/HDFS-15900
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Affects Versions: 3.3.0
>Reporter: Harunobu Daikoku
>Assignee: Harunobu Daikoku
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0
>
> Attachments: image.png
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> We observed that when a NameNode becomes UNAVAILABLE, the corresponding
> blockpool id in MembershipStoreImpl#activeNamespaces on dfsrouter
> unintentionally sets to empty, its initial value.
> !image.png|height=250!
> As a result of this, concat operations through dfsrouter fail with the
> following error as it cannot resolve the block id in the recognized active
> namespaces.
> {noformat}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RemoteException):
> Cannot locate a nameservice for block pool BP-...
> {noformat}
> A possible fix is to ignore UNAVAILABLE NameNode registrations, and set
> proper namespace information obtained from available NameNode registrations
> when constructing the cache of active namespaces.
>
> [https://github.com/apache/hadoop/blob/rel/release-3.3.0/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/impl/MembershipStoreImpl.java#L207-L221]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15900) RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode
[
https://issues.apache.org/jira/browse/HDFS-15900?focusedWorklogId=576696=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576696
]
ASF GitHub Bot logged work on HDFS-15900:
-
Author: ASF GitHub Bot
Created on: 05/Apr/21 05:04
Start Date: 05/Apr/21 05:04
Worklog Time Spent: 10m
Work Description: hdaikoku opened a new pull request #2866:
URL: https://github.com/apache/hadoop/pull/2866
https://issues.apache.org/jira/browse/HDFS-15900
Backporting https://github.com/apache/hadoop/pull/2787 to branch-3.2.
## NOTICE
Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.)
For more details, please see
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 576696)
Time Spent: 4h 10m (was: 4h)
> RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode
> ---
>
> Key: HDFS-15900
> URL: https://issues.apache.org/jira/browse/HDFS-15900
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Affects Versions: 3.3.0
>Reporter: Harunobu Daikoku
>Assignee: Harunobu Daikoku
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0
>
> Attachments: image.png
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> We observed that when a NameNode becomes UNAVAILABLE, the corresponding
> blockpool id in MembershipStoreImpl#activeNamespaces on dfsrouter
> unintentionally sets to empty, its initial value.
> !image.png|height=250!
> As a result of this, concat operations through dfsrouter fail with the
> following error as it cannot resolve the block id in the recognized active
> namespaces.
> {noformat}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RemoteException):
> Cannot locate a nameservice for block pool BP-...
> {noformat}
> A possible fix is to ignore UNAVAILABLE NameNode registrations, and set
> proper namespace information obtained from available NameNode registrations
> when constructing the cache of active namespaces.
>
> [https://github.com/apache/hadoop/blob/rel/release-3.3.0/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/impl/MembershipStoreImpl.java#L207-L221]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15950) Remove unused hdfs.proto import
[
https://issues.apache.org/jira/browse/HDFS-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314659#comment-17314659
]
Akira Ajisaka commented on HDFS-15950:
--
Note: I didn't backport this to branch-2.10 because the build failed with the
patch.
{noformat}
[INFO] --- protobuf-maven-plugin:0.5.1:compile (src-compile-protoc) @
hadoop-hdfs-client ---
[INFO] Compiling 9 proto file(s) to
/Users/aajisaka/git/github/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/target/generated-sources/java
[ERROR] PROTOC FAILED: inotify.proto:77:12: "hadoop.hdfs.FsPermissionProto"
seems to be defined in "hdfs.proto", which is not imported by "inotify.proto".
To use it here, please add the necessary import.
inotify.proto:115:12: "hadoop.hdfs.FsPermissionProto" seems to be defined in
"hdfs.proto", which is not imported by "inotify.proto". To use it here, please
add the necessary import.
ClientNamenodeProtocol.proto: Import "inotify.proto" was not found or had
errors.
ClientNamenodeProtocol.proto:747:12: "EventsListProto" is not defined.{noformat}
> Remove unused hdfs.proto import
> ---
>
> Key: HDFS-15950
> URL: https://issues.apache.org/jira/browse/HDFS-15950
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 3.4.0
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0, 3.1.5, 3.2.3
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> hdfs.proto is imported in inotify.proto and is unused. This causes the
> following warning to be generated -
> {code}
> inotify.proto:35:1: warning: Import hdfs.proto is unused.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-15950) Remove unused hdfs.proto import
[
https://issues.apache.org/jira/browse/HDFS-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved HDFS-15950.
--
Fix Version/s: 3.2.3
3.1.5
3.4.0
3.3.1
Hadoop Flags: Reviewed
Resolution: Fixed
Committed to trunk, branch-3.3, branch-3.2, and branch-3.1. Thank you
[~gautham] for the contribution!
> Remove unused hdfs.proto import
> ---
>
> Key: HDFS-15950
> URL: https://issues.apache.org/jira/browse/HDFS-15950
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 3.4.0
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0, 3.1.5, 3.2.3
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> hdfs.proto is imported in inotify.proto and is unused. This causes the
> following warning to be generated -
> {code}
> inotify.proto:35:1: warning: Import hdfs.proto is unused.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15945) DataNodes with zero capacity and zero blocks should be decommissioned immediately
[
https://issues.apache.org/jira/browse/HDFS-15945?focusedWorklogId=576685=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576685
]
ASF GitHub Bot logged work on HDFS-15945:
-
Author: ASF GitHub Bot
Created on: 05/Apr/21 02:52
Start Date: 05/Apr/21 02:52
Worklog Time Spent: 10m
Work Description: tasanuma commented on pull request #2854:
URL: https://github.com/apache/hadoop/pull/2854#issuecomment-813153233
Seems the failure of `TestDecommission.testDecommissionWithNamenodeRestart`
is related. I will investigate it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 576685)
Time Spent: 0.5h (was: 20m)
> DataNodes with zero capacity and zero blocks should be decommissioned
> immediately
> -
>
> Key: HDFS-15945
> URL: https://issues.apache.org/jira/browse/HDFS-15945
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Such as when there is a storage problem, DataNode capacity and block count
> sometimes become zero.
> When we tried to decommission those DataNodes, we ran into an issue that the
> decommission did not complete because the NameNode had not received their
> first block report.
> {noformat}
> INFO blockmanagement.DatanodeAdminManager
> (DatanodeAdminManager.java:startDecommission(183)) - Starting decommission of
> 127.0.0.1:58343
> [DISK]DS-a29de094-2b19-4834-8318-76cda3bd86bf:NORMAL:127.0.0.1:58343 with 0
> blocks
> INFO blockmanagement.BlockManager
> (BlockManager.java:isNodeHealthyForDecommissionOrMaintenance(4587)) - Node
> 127.0.0.1:58343 hasn't sent its first block report.
> INFO blockmanagement.DatanodeAdminDefaultMonitor
> (DatanodeAdminDefaultMonitor.java:check(258)) - Node 127.0.0.1:58343 isn't
> healthy. It needs to replicate 0 more blocks. Decommission In Progress is
> still in progress.
> {noformat}
> To make matters worse, even if we stopped these DataNodes afterward, they
> remained in a dead state until NameNode restarted.
> I think those DataNodes should be decommissioned immediately even if NameNode
> hasn't recived the first block report.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15950) Remove unused hdfs.proto import
[
https://issues.apache.org/jira/browse/HDFS-15950?focusedWorklogId=576684=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576684
]
ASF GitHub Bot logged work on HDFS-15950:
-
Author: ASF GitHub Bot
Created on: 05/Apr/21 02:42
Start Date: 05/Apr/21 02:42
Worklog Time Spent: 10m
Work Description: aajisaka merged pull request #2858:
URL: https://github.com/apache/hadoop/pull/2858
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 576684)
Time Spent: 40m (was: 0.5h)
> Remove unused hdfs.proto import
> ---
>
> Key: HDFS-15950
> URL: https://issues.apache.org/jira/browse/HDFS-15950
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 3.4.0
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> hdfs.proto is imported in inotify.proto and is unused. This causes the
> following warning to be generated -
> {code}
> inotify.proto:35:1: warning: Import hdfs.proto is unused.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature
[ https://issues.apache.org/jira/browse/HDFS-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314657#comment-17314657 ] Qi Zhu commented on HDFS-15601: --- [~ayushtkn] [~hexiaoqiao] Could you help review this when you are free? > Batch listing: gracefully fallback to use non-batched listing when NameNode > doesn't support the feature > --- > > Key: HDFS-15601 > URL: https://issues.apache.org/jira/browse/HDFS-15601 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chao Sun >Assignee: Qi Zhu >Priority: Major > Attachments: HDFS-15601.001.patch > > > HDFS-13616 requires both server and client side change. However, it is common > that users use a newer client to talk to older HDFS (say 2.10). Currently the > client will simply fail in this scenario. A better approach, perhaps, is to > have client fallback to use non-batched listing on the input directories. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15940) Some tests in TestBlockRecovery are consistently failing
[
https://issues.apache.org/jira/browse/HDFS-15940?focusedWorklogId=576672=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576672
]
ASF GitHub Bot logged work on HDFS-15940:
-
Author: ASF GitHub Bot
Created on: 05/Apr/21 00:49
Start Date: 05/Apr/21 00:49
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2844:
URL: https://github.com/apache/hadoop/pull/2844#issuecomment-813128479
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 35s | | trunk passed |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 21s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 52s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 13s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 11s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 55s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 55 unchanged - 7
fixed = 55 total (was 62) |
| +1 :green_heart: | mvnsite | 1m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 22s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 474m 27s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2844/13/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not
generate ASF License warnings. |
| | | 561m 25s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestDecommissioningStatus
|
| | hadoop.hdfs.TestViewDistributedFileSystemContract |
| | hadoop.hdfs.TestSnapshotCommands |
| | hadoop.hdfs.TestPersistBlocks |
| | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.TestWriteConfigurationToDFS |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestFsVolumeList |
| | hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeWithHdfsScheme |
| | hadoop.hdfs.TestStateAlignmentContextWithHA |
| |
hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeHdfsFileSystemContract |
| | hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks |
| | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| | hadoop.hdfs.server.namenode.TestNamenodeStorageDirectives |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.fs.viewfs.TestViewFSOverloadSchemeWithMountTableConfigInHDFS |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| |
hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| |
[jira] [Work logged] (HDFS-15940) Some tests in TestBlockRecovery are consistently failing
[
https://issues.apache.org/jira/browse/HDFS-15940?focusedWorklogId=576671=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576671
]
ASF GitHub Bot logged work on HDFS-15940:
-
Author: ASF GitHub Bot
Created on: 05/Apr/21 00:46
Start Date: 05/Apr/21 00:46
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2844:
URL: https://github.com/apache/hadoop/pull/2844#issuecomment-813127721
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 27s | | trunk passed |
| +1 :green_heart: | compile | 1m 21s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 1m 20s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 53s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 55 unchanged - 7
fixed = 55 total (was 62) |
| +1 :green_heart: | mvnsite | 1m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 472m 48s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2844/12/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 559m 53s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestDecommissioningStatus
|
| | hadoop.hdfs.TestViewDistributedFileSystemContract |
| | hadoop.hdfs.TestSnapshotCommands |
| | hadoop.hdfs.TestPersistBlocks |
| | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.TestWriteConfigurationToDFS |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestFsVolumeList |
| | hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeWithHdfsScheme |
| | hadoop.hdfs.TestStateAlignmentContextWithHA |
| | hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks |
| | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| | hadoop.hdfs.server.namenode.TestNamenodeStorageDirectives |
| | hadoop.fs.viewfs.TestViewFSOverloadSchemeWithMountTableConfigInHDFS |
| |
hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| | hadoop.hdfs.TestLeaseRecovery2 |
| | hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.server.datanode.TestBlockScanner |
| |
[jira] [Updated] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop
[
https://issues.apache.org/jira/browse/HDFS-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HDFS-12920:
Target Version/s: 3.0.2, 3.3.1, 3.4.0, 3.1.5, 3.2.3 (was: 3.0.2)
> HDFS default value change (with adding time unit) breaks old version MR
> tarball work with new version (3.0) of hadoop
> -
>
> Key: HDFS-12920
> URL: https://issues.apache.org/jira/browse/HDFS-12920
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Reporter: Junping Du
>Priority: Blocker
>
> After HADOOP-15059 get resolved. I tried to deploy 2.9.0 tar ball with 3.0.0
> RC1, and run the job with following errors:
> {noformat}
> 2017-12-12 13:29:06,824 INFO [main]
> org.apache.hadoop.service.AbstractService: Service
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED; cause:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.lang.NumberFormatException: For input string: "30s"
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.lang.NumberFormatException: For input string: "30s"
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:542)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:522)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1764)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:522)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:308)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1722)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1719)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1650)
> {noformat}
> This is because HDFS-10845, we are adding time unit to hdfs-default.xml but
> it cannot be recognized by old version MR jars.
> This break our rolling upgrade story, so should mark as blocker.
> A quick workaround is to add values in hdfs-site.xml with removing all time
> unit. But the right way may be to revert HDFS-10845 (and get rid of noisy
> warnings).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature
[
https://issues.apache.org/jira/browse/HDFS-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314546#comment-17314546
]
Hadoop QA commented on HDFS-15601:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 23m
21s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to
include any new or modified tests. Please justify why no new tests are needed
for this patch. Also please list what manual steps were performed to verify
this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
39s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 19s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 19m
58s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 2m
29s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 18s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/564/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-client.txt{color}
| {color:orange} hadoop-hdfs-project/hadoop-hdfs-client: The patch generated 1
new + 18 unchanged - 0 fixed = 19 total (was 18) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 54s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts.
[jira] [Work logged] (HDFS-15916) DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff
[
https://issues.apache.org/jira/browse/HDFS-15916?focusedWorklogId=576591=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576591
]
ASF GitHub Bot logged work on HDFS-15916:
-
Author: ASF GitHub Bot
Created on: 04/Apr/21 16:02
Start Date: 04/Apr/21 16:02
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2863:
URL: https://github.com/apache/hadoop/pull/2863#issuecomment-813057205
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 21m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 53s | | trunk passed |
| +1 :green_heart: | compile | 0m 58s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 48s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 55s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 49s | | the patch passed |
| +1 :green_heart: | compile | 0m 52s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 52s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 42s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 13s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 107m 2s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2863/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2863 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux c946ba7bb398 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9
06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a9be8aa30ddae1654cd40c5ce68a91a2d5a6077c |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2863/1/testReport/ |
| Max. process+thread count | 569 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client U:
hadoop-hdfs-project/hadoop-hdfs-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2863/1/console |
|
[jira] [Commented] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature
[ https://issues.apache.org/jira/browse/HDFS-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314516#comment-17314516 ] Qi Zhu commented on HDFS-15601: --- I will take it, since still no one take. > Batch listing: gracefully fallback to use non-batched listing when NameNode > doesn't support the feature > --- > > Key: HDFS-15601 > URL: https://issues.apache.org/jira/browse/HDFS-15601 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chao Sun >Assignee: Qi Zhu >Priority: Major > Attachments: HDFS-15601.001.patch > > > HDFS-13616 requires both server and client side change. However, it is common > that users use a newer client to talk to older HDFS (say 2.10). Currently the > client will simply fail in this scenario. A better approach, perhaps, is to > have client fallback to use non-batched listing on the input directories. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature
[ https://issues.apache.org/jira/browse/HDFS-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated HDFS-15601: -- Attachment: HDFS-15601.001.patch Status: Patch Available (was: Open) > Batch listing: gracefully fallback to use non-batched listing when NameNode > doesn't support the feature > --- > > Key: HDFS-15601 > URL: https://issues.apache.org/jira/browse/HDFS-15601 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chao Sun >Assignee: Qi Zhu >Priority: Major > Attachments: HDFS-15601.001.patch > > > HDFS-13616 requires both server and client side change. However, it is common > that users use a newer client to talk to older HDFS (say 2.10). Currently the > client will simply fail in this scenario. A better approach, perhaps, is to > have client fallback to use non-batched listing on the input directories. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature
[ https://issues.apache.org/jira/browse/HDFS-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reassigned HDFS-15601: - Assignee: Qi Zhu > Batch listing: gracefully fallback to use non-batched listing when NameNode > doesn't support the feature > --- > > Key: HDFS-15601 > URL: https://issues.apache.org/jira/browse/HDFS-15601 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Reporter: Chao Sun >Assignee: Qi Zhu >Priority: Major > > HDFS-13616 requires both server and client side change. However, it is common > that users use a newer client to talk to older HDFS (say 2.10). Currently the > client will simply fail in this scenario. A better approach, perhaps, is to > have client fallback to use non-batched listing on the input directories. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15940) Some tests in TestBlockRecovery are consistently failing
[ https://issues.apache.org/jira/browse/HDFS-15940?focusedWorklogId=576580=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576580 ] ASF GitHub Bot logged work on HDFS-15940: - Author: ASF GitHub Bot Created on: 04/Apr/21 14:28 Start Date: 04/Apr/21 14:28 Worklog Time Spent: 10m Work Description: ayushtkn commented on pull request #2844: URL: https://github.com/apache/hadoop/pull/2844#issuecomment-813042644 There are checkstyle warnings from the new file, https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2844/8/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt These we need to fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 576580) Time Spent: 2h 50m (was: 2h 40m) > Some tests in TestBlockRecovery are consistently failing > > > Key: HDFS-15940 > URL: https://issues.apache.org/jira/browse/HDFS-15940 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.4.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > Some long running tests in TestBlockRecovery are consistently failing. Also, > TestBlockRecovery is huge with so many tests, we should refactor some of long > running and race condition specific tests to separate class. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15916) DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff
[
https://issues.apache.org/jira/browse/HDFS-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314506#comment-17314506
]
Ayush Saxena commented on HDFS-15916:
-
Raised a PR, Going backward would have a couple of challenges, The snapshot
diff one should be solved by this.
HDFS-12920 might surface as well, at least it did for me, in that case remove
the time suffix from {{dfs.client.datanode-restart.timeout}}
In case you want to try, then try from below 3.3, from 3.3 and above, there
might be an error due to {{ClassNotFound}} for {{ProtobuffRpcEngine2}}
> DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for
> snapshotdiff
> ---
>
> Key: HDFS-15916
> URL: https://issues.apache.org/jira/browse/HDFS-15916
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.2.2
>Reporter: Srinivasu Majeti
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Looks like when using distcp diff options between two snapshots from a hadoop
> 3 cluster to hadoop 2 cluster , we get below exception and seems to be break
> backward compatibility due to new API introduction
> getSnapshotDiffReportListing.
>
> {code:java}
> hadoop distcp -diff s1 s2 -update src_cluster_path dst_cluster_path
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RpcNoSuchMethodException):
> Unknown method getSnapshotDiffReportListing called on
> org.apache.hadoop.hdfs.protocol.ClientProtocol protocol
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15916) DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff
[
https://issues.apache.org/jira/browse/HDFS-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDFS-15916:
--
Labels: pull-request-available (was: )
> DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for
> snapshotdiff
> ---
>
> Key: HDFS-15916
> URL: https://issues.apache.org/jira/browse/HDFS-15916
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.2.2
>Reporter: Srinivasu Majeti
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Looks like when using distcp diff options between two snapshots from a hadoop
> 3 cluster to hadoop 2 cluster , we get below exception and seems to be break
> backward compatibility due to new API introduction
> getSnapshotDiffReportListing.
>
> {code:java}
> hadoop distcp -diff s1 s2 -update src_cluster_path dst_cluster_path
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RpcNoSuchMethodException):
> Unknown method getSnapshotDiffReportListing called on
> org.apache.hadoop.hdfs.protocol.ClientProtocol protocol
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15916) DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff
[
https://issues.apache.org/jira/browse/HDFS-15916?focusedWorklogId=576578=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576578
]
ASF GitHub Bot logged work on HDFS-15916:
-
Author: ASF GitHub Bot
Created on: 04/Apr/21 14:13
Start Date: 04/Apr/21 14:13
Worklog Time Spent: 10m
Work Description: ayushtkn opened a new pull request #2863:
URL: https://github.com/apache/hadoop/pull/2863
https://issues.apache.org/jira/browse/HDFS-15916
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 576578)
Remaining Estimate: 0h
Time Spent: 10m
> DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for
> snapshotdiff
> ---
>
> Key: HDFS-15916
> URL: https://issues.apache.org/jira/browse/HDFS-15916
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.2.2
>Reporter: Srinivasu Majeti
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Looks like when using distcp diff options between two snapshots from a hadoop
> 3 cluster to hadoop 2 cluster , we get below exception and seems to be break
> backward compatibility due to new API introduction
> getSnapshotDiffReportListing.
>
> {code:java}
> hadoop distcp -diff s1 s2 -update src_cluster_path dst_cluster_path
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RpcNoSuchMethodException):
> Unknown method getSnapshotDiffReportListing called on
> org.apache.hadoop.hdfs.protocol.ClientProtocol protocol
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15916) DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff
[
https://issues.apache.org/jira/browse/HDFS-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HDFS-15916:
Summary: DistCp: Backward compatibility: Distcp fails from Hadoop 3 to
Hadoop 2 for snapshotdiff (was: Backward compatibility - Distcp fails from
Hadoop 3 to Hadoop 2 for snapshotdiff)
> DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for
> snapshotdiff
> ---
>
> Key: HDFS-15916
> URL: https://issues.apache.org/jira/browse/HDFS-15916
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.2.2
>Reporter: Srinivasu Majeti
>Priority: Major
>
> Looks like when using distcp diff options between two snapshots from a hadoop
> 3 cluster to hadoop 2 cluster , we get below exception and seems to be break
> backward compatibility due to new API introduction
> getSnapshotDiffReportListing.
>
> {code:java}
> hadoop distcp -diff s1 s2 -update src_cluster_path dst_cluster_path
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RpcNoSuchMethodException):
> Unknown method getSnapshotDiffReportListing called on
> org.apache.hadoop.hdfs.protocol.ClientProtocol protocol
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15940) Some tests in TestBlockRecovery are consistently failing
[
https://issues.apache.org/jira/browse/HDFS-15940?focusedWorklogId=576574=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576574
]
ASF GitHub Bot logged work on HDFS-15940:
-
Author: ASF GitHub Bot
Created on: 04/Apr/21 13:23
Start Date: 04/Apr/21 13:23
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2844:
URL: https://github.com/apache/hadoop/pull/2844#issuecomment-813032279
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 1s | | trunk passed |
| +1 :green_heart: | compile | 1m 33s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 18s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 21s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 19m 38s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 27s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 27s | | the patch passed |
| +1 :green_heart: | compile | 1m 12s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 57s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2844/8/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 55 unchanged
- 7 fixed = 65 total (was 62) |
| +1 :green_heart: | mvnsite | 1m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 359m 27s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2844/8/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 456m 3s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestBootstrapStandby |
| | hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots |
| | hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.TestLeaseRecovery2 |
| | hadoop.hdfs.TestPersistBlocks |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.TestSnapshotCommands |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| |
hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestFsVolumeList |
| |
[jira] [Work logged] (HDFS-15940) Some tests in TestBlockRecovery are consistently failing
[
https://issues.apache.org/jira/browse/HDFS-15940?focusedWorklogId=576571=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576571
]
ASF GitHub Bot logged work on HDFS-15940:
-
Author: ASF GitHub Bot
Created on: 04/Apr/21 13:20
Start Date: 04/Apr/21 13:20
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2844:
URL: https://github.com/apache/hadoop/pull/2844#issuecomment-813031835
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 20m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 53s | | trunk passed |
| +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 29s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 15s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 19m 32s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 26s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 55s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2844/7/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 55 unchanged
- 7 fixed = 65 total (was 62) |
| +1 :green_heart: | mvnsite | 1m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 42s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 11s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 357m 4s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2844/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 18s | | The patch does not
generate ASF License warnings. |
| | | 473m 43s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestBootstrapStandby |
| | hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots |
| | hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks |
| | hadoop.hdfs.server.datanode.TestIncrementalBrVariations |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.TestLeaseRecovery2 |
| | hadoop.hdfs.TestPersistBlocks |
| | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.TestSnapshotCommands |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| |
hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| |
[jira] [Work logged] (HDFS-15940) Some tests in TestBlockRecovery are consistently failing
[
https://issues.apache.org/jira/browse/HDFS-15940?focusedWorklogId=576543=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576543
]
ASF GitHub Bot logged work on HDFS-15940:
-
Author: ASF GitHub Bot
Created on: 04/Apr/21 09:22
Start Date: 04/Apr/21 09:22
Worklog Time Spent: 10m
Work Description: virajjasani commented on a change in pull request #2844:
URL: https://github.com/apache/hadoop/pull/2844#discussion_r606773745
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery1.java
##
@@ -336,191 +322,194 @@ public void testFinalizedReplicas () throws IOException
{
if(LOG.isDebugEnabled()) {
LOG.debug("Running " + GenericTestUtils.getMethodName());
}
-ReplicaRecoveryInfo replica1 = new ReplicaRecoveryInfo(BLOCK_ID,
-REPLICA_LEN1, GEN_STAMP-1, ReplicaState.FINALIZED);
-ReplicaRecoveryInfo replica2 = new ReplicaRecoveryInfo(BLOCK_ID,
-REPLICA_LEN1, GEN_STAMP-2, ReplicaState.FINALIZED);
+ReplicaRecoveryInfo replica1 = new ReplicaRecoveryInfo(BLOCK_ID,
+REPLICA_LEN1, GEN_STAMP - 1, ReplicaState.FINALIZED);
+ReplicaRecoveryInfo replica2 = new ReplicaRecoveryInfo(BLOCK_ID,
+REPLICA_LEN1, GEN_STAMP - 2, ReplicaState.FINALIZED);
Review comment:
Oh you are right, maybe QA bot had EOL issues which were not relevant to
actual changes because I renamed the file earlier.
> Anyway lets ignore for now. I expect the build won't have anything which
would require you any more commits
Sure sounds good, I will take care of this going forward. It's the rename
which might let QA bot scan entire file regardless of how minor the actual
change is in that file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 576543)
Time Spent: 2h 20m (was: 2h 10m)
> Some tests in TestBlockRecovery are consistently failing
>
>
> Key: HDFS-15940
> URL: https://issues.apache.org/jira/browse/HDFS-15940
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Some long running tests in TestBlockRecovery are consistently failing. Also,
> TestBlockRecovery is huge with so many tests, we should refactor some of long
> running and race condition specific tests to separate class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15940) Some tests in TestBlockRecovery are consistently failing
[
https://issues.apache.org/jira/browse/HDFS-15940?focusedWorklogId=576540=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576540
]
ASF GitHub Bot logged work on HDFS-15940:
-
Author: ASF GitHub Bot
Created on: 04/Apr/21 09:12
Start Date: 04/Apr/21 09:12
Worklog Time Spent: 10m
Work Description: ayushtkn commented on a change in pull request #2844:
URL: https://github.com/apache/hadoop/pull/2844#discussion_r606772623
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery1.java
##
@@ -336,191 +322,194 @@ public void testFinalizedReplicas () throws IOException
{
if(LOG.isDebugEnabled()) {
LOG.debug("Running " + GenericTestUtils.getMethodName());
}
-ReplicaRecoveryInfo replica1 = new ReplicaRecoveryInfo(BLOCK_ID,
-REPLICA_LEN1, GEN_STAMP-1, ReplicaState.FINALIZED);
-ReplicaRecoveryInfo replica2 = new ReplicaRecoveryInfo(BLOCK_ID,
-REPLICA_LEN1, GEN_STAMP-2, ReplicaState.FINALIZED);
+ReplicaRecoveryInfo replica1 = new ReplicaRecoveryInfo(BLOCK_ID,
+REPLICA_LEN1, GEN_STAMP - 1, ReplicaState.FINALIZED);
+ReplicaRecoveryInfo replica2 = new ReplicaRecoveryInfo(BLOCK_ID,
+REPLICA_LEN1, GEN_STAMP - 2, ReplicaState.FINALIZED);
Review comment:
May be because you changed the file name and it read it as a new file?
one or two fixes are ok, but in case the PR doesn't induce the checkstyle
and there are plenty. we can let it stay as is, and let the one approving spend
some time, checking that. :-)
Anyway lets ignore for now. I expect the build won't have anything which
would require you any more commits
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 576540)
Time Spent: 2h 10m (was: 2h)
> Some tests in TestBlockRecovery are consistently failing
>
>
> Key: HDFS-15940
> URL: https://issues.apache.org/jira/browse/HDFS-15940
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Some long running tests in TestBlockRecovery are consistently failing. Also,
> TestBlockRecovery is huge with so many tests, we should refactor some of long
> running and race condition specific tests to separate class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
