[jira] [Work logged] (HDFS-16208) [FGL] Implement Delete API with FGL
[
https://issues.apache.org/jira/browse/HDFS-16208?focusedWorklogId=796227=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796227
]
ASF GitHub Bot logged work on HDFS-16208:
-
Author: ASF GitHub Bot
Created on: 29/Jul/22 03:20
Start Date: 29/Jul/22 03:20
Worklog Time Spent: 10m
Work Description: jianghuazhu commented on PR #3376:
URL: https://github.com/apache/hadoop/pull/3376#issuecomment-1198838825
Thanks @prasad-acit .
I have read your update carefully, here are some of my personal insights:
1. We should hold child locks in FSDirDeleteOp, here are some of my
suggestions.
`FSDirDeleteOp#deleteInternal() {
''
fsd.getINodeMap().latchWriteLock(iip.getExistingINodes(), new INode[]{});
fsn.removeLeasesAndINodes(removedUCFiles, removedINodes, true);
''
}
`
At this stage, the deleteInternal() operation may be safer, and when FGL
becomes more mature, it can be processed more fine-grained.
2. Adding hasWriteLock() to PartitionedGSet may still fail to accurately
capture each Partition and lock. Because the elements saved in PartitionedGSet
will be repeated and distributed in different Partitions.
3. It seems unnecessary to add hasWriteLock() and latchWriteLock() to
INodeMap because latchWriteLock(INodesInPath iip, INode[] missing) already
exists.
Also, regarding the child lock issue you mentioned, I think it's a matter of
concern. I'll submit some PRs later.
Welcome to the discussion, @prasad-acit .
Issue Time Tracking
---
Worklog Id: (was: 796227)
Time Spent: 1.5h (was: 1h 20m)
> [FGL] Implement Delete API with FGL
> ---
>
> Key: HDFS-16208
> URL: https://issues.apache.org/jira/browse/HDFS-16208
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Renukaprasad C
>Assignee: Renukaprasad C
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Replace all global locks for file / directory deletion with FGL.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11440) Error compiling native libraries error: ^ ^ _mm_set_pd ^ ^ : target specific option mismatch
[ https://issues.apache.org/jira/browse/HDFS-11440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572656#comment-17572656 ] Steve Loughran commented on HDFS-11440: --- is this still an issue or can we close it? > Error compiling native libraries error: ^ ^ _mm_set_pd ^ ^ : target specific > option mismatch > > > Key: HDFS-11440 > URL: https://issues.apache.org/jira/browse/HDFS-11440 > Project: Hadoop HDFS > Issue Type: Bug > Components: build >Affects Versions: 2.7.3 > Environment: Linux 3.16.0-4-686-pae #1 SMP Debian 3.16.39-1 > (2016-12-30) i686 GNU/Linux > Machine: System: Hewlett-Packard product: HP Compaq dc7100 SFF(PC922A) > serial: MXJ512018N >Mobo: Hewlett-Packard model: 097Ch serial: MXJ512018N >Bios: Hewlett-Packard v: 786C1 v01.05 date: 06/16/2004 >Reporter: Dario Castaño >Priority: Blocker > Labels: build > > When i try to build native libraries for my system, i'm getting this error: > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 7:48.369s > [INFO] Finished at: Wed Feb 22 13:14:21 EST 2017 > [INFO] Final Memory: 69M/247M > [INFO] > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project > hadoop-hdfs: An Ant BuildException has occured: exec returned: 1 > [ERROR] around Ant part ... dir="/home/hadoop/hadoop-2.7.3/src/hadoop-2.7.3-src/hadoop-hdfs-project/hadoop-hdfs/target/native" > executable="cmake">... @ 5:146 in > /home/hadoop/hadoop-2.7.3/src/hadoop-2.7.3-src/hadoop-hdfs-project/hadoop-hdfs/target/antrun/build-main.xml > [ERROR] -> [Help 1] > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project > hadoop-hdfs: An Ant BuildException has occured: exec returned: 1 > around Ant part ... dir="/home/hadoop/hadoop-2.7.3/src/hadoop-2.7.3-src/hadoop-hdfs-project/hadoop-hdfs/target/native" > executable="cmake">... @ 5:146 in > /home/hadoop/hadoop-2.7.3/src/hadoop-2.7.3-src/hadoop-hdfs-project/hadoop-hdfs/target/antrun/build-main.xml > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59) > at > org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183) > at > org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161) > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320) > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) > at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537) > at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196) > at org.apache.maven.cli.MavenCli.main(MavenCli.java:141) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) > at > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) > at > org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) > Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant > BuildException has occured: exec returned: 1 > around Ant part ... dir="/home/hadoop/hadoop-2.7.3/src/hadoop-2.7.3-src/hadoop-hdfs-project/hadoop-hdfs/target/native" > executable="cmake">... @ 5:146 in > /home/hadoop/hadoop-2.7.3/src/hadoop-2.7.3-src/hadoop-hdfs-project/hadoop-hdfs/target/antrun/build-main.xml > at > org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355) > at > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) > at >
[jira] [Resolved] (HDFS-9475) execution of org.apache.hadoop.hdfs.net.TcpPeerServer.close() causes timeout on Hadoop-2.6.0 with IBM-JDK-1.8
[ https://issues.apache.org/jira/browse/HDFS-9475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HDFS-9475. -- Resolution: Won't Fix > execution of org.apache.hadoop.hdfs.net.TcpPeerServer.close() causes timeout > on Hadoop-2.6.0 with IBM-JDK-1.8 > - > > Key: HDFS-9475 > URL: https://issues.apache.org/jira/browse/HDFS-9475 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.6.0, 2.7.1 > Environment: IBM JDK 1.8.0 > Architecture: s390x GNU/Linux >Reporter: Rakesh Sharma >Priority: Blocker > > --- > Test set: org.apache.hadoop.hdfs.server.balancer.TestBalancer > --- > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 101.69 sec > <<< FAILURE! - in org.apache.hadoop.hdfs.server.balancer.TestBalancer > testTwoReplicaShouldNotInSameDN(org.apache.hadoop.hdfs.server.balancer.TestBalancer) > Time elapsed: 100.008 sec <<< ERROR! > java.lang.Exception: test timed out after 10 milliseconds > at > java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:245) > at > java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:126) > at sun.nio.ch.ServerSocketAdaptor.close(ServerSocketAdaptor.java:149) > at > org.apache.hadoop.hdfs.net.TcpPeerServer.close(TcpPeerServer.java:153) > at > org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.kill(DataXceiverServer.java:223) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.shutdown(DataNode.java:1663) > at > org.apache.hadoop.hdfs.MiniDFSCluster.shutdownDataNodes(MiniDFSCluster.java:1750) > at > org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1721) > at > org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705) > at > org.apache.hadoop.hdfs.server.balancer.TestBalancer.testTwoReplicaShouldNotInSameDN(TestBalancer.java:1382) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16678) RBF supports disable getNodeUsage() in RBFMetrics
[
https://issues.apache.org/jira/browse/HDFS-16678?focusedWorklogId=796166=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796166
]
ASF GitHub Bot logged work on HDFS-16678:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 18:52
Start Date: 28/Jul/22 18:52
Worklog Time Spent: 10m
Work Description: goiri commented on code in PR #4606:
URL: https://github.com/apache/hadoop/pull/4606#discussion_r932570164
##
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/RBFMetrics.java:
##
@@ -537,35 +547,34 @@ public int getNumEnteringMaintenanceDataNodes() {
@Override // NameNodeMXBean
public String getNodeUsage() {
-float median = 0;
-float max = 0;
-float min = 0;
-float dev = 0;
+double median = 0;
+double max = 0;
+double min = 0;
+double dev = 0;
final Map> info = new HashMap<>();
try {
- RouterRpcServer rpcServer = this.router.getRpcServer();
- DatanodeInfo[] live = rpcServer.getDatanodeReport(
- DatanodeReportType.LIVE, false, timeOut);
+ DatanodeInfo[] live = null;
+ if (this.enableGetDNUsage) {
+RouterRpcServer rpcServer = this.router.getRpcServer();
+live = rpcServer.getDatanodeReport(DatanodeReportType.LIVE, false,
timeOut);
+ } else {
+LOG.debug("Getting node usage is disabled.");
+ }
- if (live.length > 0) {
-float totalDfsUsed = 0;
-float[] usages = new float[live.length];
+ if (live != null && live.length > 0) {
+double[] usages = new double[live.length];
int i = 0;
for (DatanodeInfo dn : live) {
usages[i++] = dn.getDfsUsedPercent();
- totalDfsUsed += dn.getDfsUsedPercent();
}
-totalDfsUsed /= live.length;
Arrays.sort(usages);
median = usages[usages.length / 2];
max = usages[usages.length - 1];
min = usages[0];
-for (i = 0; i < usages.length; i++) {
- dev += (usages[i] - totalDfsUsed) * (usages[i] - totalDfsUsed);
-}
-dev = (float) Math.sqrt(dev / usages.length);
+StandardDeviation deviation = new StandardDeviation();
+dev = deviation.evaluate(usages);
}
} catch (IOException e) {
LOG.error("Cannot get the live nodes: {}", e.getMessage());
Review Comment:
> I feel it would be better this way.
>
> ```
> LOG.error("Cannot get the live nodes.", e).
> ```
Do we want to have the full stack trace? I think it is pretty clear what the
error is here without it.
Issue Time Tracking
---
Worklog Id: (was: 796166)
Time Spent: 1h 50m (was: 1h 40m)
> RBF supports disable getNodeUsage() in RBFMetrics
> -
>
> Key: HDFS-16678
> URL: https://issues.apache.org/jira/browse/HDFS-16678
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> In our prod environment, we try to collect RBF metrics every 15s through
> jmx_exporter. And we found that collection task often failed.
> After tracing and found that the collection task is blocked at getNodeUsage()
> in RBFMetrics, because it will collection all datanode's usage from
> downstream nameservices. This is a very expensive and almost useless
> operation. Because in most scenarios, each NameSerivce contains almost the
> same DNs. We can get the data usage's from any one nameservices, not from RBF.
> So I feel that RBF should supports disable getNodeUsage() in RBFMetrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16671) RBF: RouterRpcFairnessPolicyController supports configurable permit acquire timeout
[
https://issues.apache.org/jira/browse/HDFS-16671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri updated HDFS-16671:
---
Fix Version/s: 3.4.0
(was: 1.3.0)
> RBF: RouterRpcFairnessPolicyController supports configurable permit acquire
> timeout
> ---
>
> Key: HDFS-16671
> URL: https://issues.apache.org/jira/browse/HDFS-16671
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> RouterRpcFairnessPolicyController supports configurable permit acquire
> timeout. Hardcode 1s is very long, and it has causes an incident in our prod
> environment when one nameserivce is busy.
> And the optimal timeout maybe should be less than p50(avgTime).
> And all handlers in RBF is waiting to acquire the permit of the busy ns.
> {code:java}
> "IPC Server handler 12 on default port " #2370 daemon prio=5 os_prio=0
> tid=? nid=? waiting on condition [?]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for (a
> java.util.concurrent.Semaphore$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:409)
> at
> org.apache.hadoop.hdfs.server.federation.fairness.AbstractRouterRpcFairnessPolicyController.acquirePermit(AbstractRouterRpcFairnessPolicyController.java:56)
> at
> org.apache.hadoop.hdfs.server.federation.fairness.DynamicRouterRpcFairnessPolicyController.acquirePermit(DynamicRouterRpcFairnessPolicyController.java:123)
> at
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.acquirePermit(RouterRpcClient.java:1500)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13522) RBF: Support observer node from Router-Based Federation
[
https://issues.apache.org/jira/browse/HDFS-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572562#comment-17572562
]
Erik Krogen commented on HDFS-13522:
Thanks for sharing [~xuzq_zander], very interesting!
{quote}
I just feel that Design A will caused Client carries a lot of useless NS TxIds
to RBF, because at most scenarios, RBF just proxy request from one client to
one downstream NS. And as the underlying NS increases, client will carries more
and more useless NS TxIds to RBF.
{quote}
[~simbadzina] -- I haven't looked carefully at the design but are we currently
including all downstream NS in the header regardless of whether or not a client
accesses it? It's a good point that ideally we would only include the state for
NS which are actually accessed.
> RBF: Support observer node from Router-Based Federation
> ---
>
> Key: HDFS-13522
> URL: https://issues.apache.org/jira/browse/HDFS-13522
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: federation, namenode
>Reporter: Erik Krogen
>Assignee: Simbarashe Dzinamarira
>Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-13522.001.patch, HDFS-13522.002.patch,
> HDFS-13522_WIP.patch, HDFS-13522_proposal_zhengchenyu_v1.pdf, RBF_ Observer
> support.pdf, Router+Observer RPC clogging.png,
> ShortTerm-Routers+Observer.png,
> observer_reads_in_rbf_proposal_simbadzina_v1.pdf,
> observer_reads_in_rbf_proposal_simbadzina_v2.pdf
>
> Time Spent: 20h 50m
> Remaining Estimate: 0h
>
> Changes will need to occur to the router to support the new observer node.
> One such change will be to make the router understand the observer state,
> e.g. {{FederationNamenodeServiceState}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13522) RBF: Support observer node from Router-Based Federation
[
https://issues.apache.org/jira/browse/HDFS-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572562#comment-17572562
]
Erik Krogen edited comment on HDFS-13522 at 7/28/22 5:01 PM:
-
Thanks for sharing [~xuzq_zander], very interesting!
{quote}
I just feel that Design A will caused Client carries a lot of useless NS TxIds
to RBF, because at most scenarios, RBF just proxy request from one client to
one downstream NS. And as the underlying NS increases, client will carries more
and more useless NS TxIds to RBF.
{quote}
[~simbadzina] -- I haven't looked carefully at the design but are we currently
including all downstream NS in the header regardless of whether or not a client
accesses it? It's a good point that ideally we would only include the state for
NS which are actually accessed (even if there are only single-digit NS, still
there's no point carrying the extra state if it is not used).
was (Author: xkrogen):
Thanks for sharing [~xuzq_zander], very interesting!
{quote}
I just feel that Design A will caused Client carries a lot of useless NS TxIds
to RBF, because at most scenarios, RBF just proxy request from one client to
one downstream NS. And as the underlying NS increases, client will carries more
and more useless NS TxIds to RBF.
{quote}
[~simbadzina] -- I haven't looked carefully at the design but are we currently
including all downstream NS in the header regardless of whether or not a client
accesses it? It's a good point that ideally we would only include the state for
NS which are actually accessed.
> RBF: Support observer node from Router-Based Federation
> ---
>
> Key: HDFS-13522
> URL: https://issues.apache.org/jira/browse/HDFS-13522
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: federation, namenode
>Reporter: Erik Krogen
>Assignee: Simbarashe Dzinamarira
>Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-13522.001.patch, HDFS-13522.002.patch,
> HDFS-13522_WIP.patch, HDFS-13522_proposal_zhengchenyu_v1.pdf, RBF_ Observer
> support.pdf, Router+Observer RPC clogging.png,
> ShortTerm-Routers+Observer.png,
> observer_reads_in_rbf_proposal_simbadzina_v1.pdf,
> observer_reads_in_rbf_proposal_simbadzina_v2.pdf
>
> Time Spent: 20h 50m
> Remaining Estimate: 0h
>
> Changes will need to occur to the router to support the new observer node.
> One such change will be to make the router understand the observer state,
> e.g. {{FederationNamenodeServiceState}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-13522) RBF: Support observer node from Router-Based Federation
[
https://issues.apache.org/jira/browse/HDFS-13522?focusedWorklogId=796096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796096
]
ASF GitHub Bot logged work on HDFS-13522:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 15:14
Start Date: 28/Jul/22 15:14
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4127:
URL: https://github.com/apache/hadoop/pull/4127#issuecomment-1198294890
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 14s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 27s | | trunk passed |
| +1 :green_heart: | compile | 23m 8s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 52s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 7m 33s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 35s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 6m 56s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 12m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 29s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 11s | | the patch passed |
| +1 :green_heart: | compile | 22m 28s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 22m 28s | | the patch passed |
| +1 :green_heart: | javac | 22m 28s | | the patch passed |
| +1 :green_heart: | compile | 20m 49s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 20m 49s | | the patch passed |
| +1 :green_heart: | javac | 20m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 21s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/27/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 5 new + 320 unchanged - 1 fixed = 325 total (was
321) |
| +1 :green_heart: | mvnsite | 7m 49s | | the patch passed |
| +1 :green_heart: | javadoc | 6m 3s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 6m 55s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 13m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 20m 11s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 19s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 401m 11s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | unit | 39m 41s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 48s | | The patch does not
generate ASF License warnings. |
| | | 727m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/27/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4127 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit
[jira] [Work logged] (HDFS-16697) Randomly setting “dfs.namenode.resource.checked.volumes.minimum” will always prevent safe mode from being turned off
[
https://issues.apache.org/jira/browse/HDFS-16697?focusedWorklogId=796070=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796070
]
ASF GitHub Bot logged work on HDFS-16697:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 13:30
Start Date: 28/Jul/22 13:30
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4653:
URL: https://github.com/apache/hadoop/pull/4653#issuecomment-1198144710
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 59s | | trunk passed |
| +1 :green_heart: | compile | 1m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 18s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 44s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 26s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/2/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch failed. |
| -1 :x: | compile | 0m 26s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/2/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| -1 :x: | javac | 0m 26s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/2/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| -1 :x: | compile | 0m 26s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/2/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
| -1 :x: | javac | 0m 26s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/2/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/2/artifact/out/blanks-eol.txt)
| The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 1s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/2/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 8 unchanged -
0 fixed = 12 total (was 8) |
| -1 :x: | mvnsite | 0m 27s |
[jira] [Work logged] (HDFS-13522) RBF: Support observer node from Router-Based Federation
[
https://issues.apache.org/jira/browse/HDFS-13522?focusedWorklogId=796054=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796054
]
ASF GitHub Bot logged work on HDFS-13522:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 12:18
Start Date: 28/Jul/22 12:18
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4311:
URL: https://github.com/apache/hadoop/pull/4311#issuecomment-1198065242
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 34s | | trunk passed |
| +1 :green_heart: | compile | 23m 33s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 47s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 7m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 33s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 6m 49s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 12m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 13s | | the patch passed |
| +1 :green_heart: | compile | 22m 34s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 22m 34s | | the patch passed |
| +1 :green_heart: | javac | 22m 34s | | the patch passed |
| +1 :green_heart: | compile | 20m 57s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 20m 57s | | the patch passed |
| +1 :green_heart: | javac | 20m 57s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 20s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4311/19/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 177 unchanged - 1 fixed = 181 total (was
178) |
| +1 :green_heart: | mvnsite | 7m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 6m 20s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 6m 45s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 12m 55s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 49s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 49s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 18s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 242m 58s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | unit | 23m 15s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 58s | | The patch does not
generate ASF License warnings. |
| | | 551m 16s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4311/19/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4311 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint
bufcompat |
|
[jira] [Updated] (HDFS-16482) ObserverNamenode throw FileNotFoundException on addBlock
[
https://issues.apache.org/jira/browse/HDFS-16482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
cp updated HDFS-16482:
--
Description:
DfsClient call ObsererNamenode throw FileNotFoundException:
{code:java}
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2898)
at
org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.analyzeFileState(FSDirWriteFileOp.java:599)
at
org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.validateAddBlock(FSDirWriteFileOp.java:171)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2777)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:892)
{code}
addBlock operation should send to ANN，but if ANN is changed or access ANN
timeout，DfsClient will find ANN from failOverProxy which ObserverNode in. If
the command send to ObserverNode, ObserverNode will not block until it is
synchronized with the client because the method is not readOnly.
!image-2022-03-25-18-21-57-677.png!
getAdditionalBlock method check read operation and check file exits or not and
check write operation finally.
!image-2022-03-25-18-25-19-808.png|width=230,height=266!
I think the ObserverNameNode should not handle this method firstly
was:
DfsClient call ObsererNamenode throw FileNotFoundException:
{code:java}
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2898)
at
org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.analyzeFileState(FSDirWriteFileOp.java:599)
at
org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.validateAddBlock(FSDirWriteFileOp.java:171)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2777)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:892)
{code}
addBlock operation should send to ANN，but if ANN is changed or access ANN
timeout，DfsClient will find ANN from failOverProxy which ObserverNode in. If
the command send to ObserverNode, ObserverNode will not block until it is
synchronized with the client because the method is not readOnly.
!image-2022-03-25-18-21-57-677.png!
getAdditionalBlock method check read operation and check file exits or not and
check write operation finally.
!image-2022-03-25-18-25-19-808.png!
I think the ObserverNameNode should not handle this method firstly
> ObserverNamenode throw FileNotFoundException on addBlock
> -
>
> Key: HDFS-16482
> URL: https://issues.apache.org/jira/browse/HDFS-16482
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: dfsclient, namenode
>Affects Versions: 3.2.1
>Reporter: cp
>Priority: Major
> Attachments: image-2022-03-25-18-18-22-720.png,
> image-2022-03-25-18-25-19-808.png
>
>
> DfsClient call ObsererNamenode throw FileNotFoundException:
> {code:java}
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2898)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.analyzeFileState(FSDirWriteFileOp.java:599)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.validateAddBlock(FSDirWriteFileOp.java:171)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2777)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:892)
> {code}
>
> addBlock operation should send to ANN，but if ANN is changed or access ANN
> timeout，DfsClient will find ANN from failOverProxy which ObserverNode in. If
> the command send to ObserverNode, ObserverNode will not block until it is
> synchronized with the client because the method is not readOnly.
> !image-2022-03-25-18-21-57-677.png!
> getAdditionalBlock method check read operation and check file exits or not
> and check write operation finally.
> !image-2022-03-25-18-25-19-808.png|width=230,height=266!
> I think the ObserverNameNode should not handle this method firstly
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-13522) RBF: Support observer node from Router-Based Federation
[
https://issues.apache.org/jira/browse/HDFS-13522?focusedWorklogId=796024=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796024
]
ASF GitHub Bot logged work on HDFS-13522:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 10:22
Start Date: 28/Jul/22 10:22
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4127:
URL: https://github.com/apache/hadoop/pull/4127#issuecomment-1197953939
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 4s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 14s | | trunk passed |
| +1 :green_heart: | compile | 26m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 24m 57s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 5m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 24s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 5m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 11m 52s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 4s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 5s | | the patch passed |
| +1 :green_heart: | compile | 24m 17s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 24m 17s | | the patch passed |
| +1 :green_heart: | javac | 24m 17s | | the patch passed |
| +1 :green_heart: | compile | 21m 41s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 21m 41s | | the patch passed |
| +1 :green_heart: | javac | 21m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 28s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/26/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 5 new + 320 unchanged - 1 fixed = 325 total (was
321) |
| +1 :green_heart: | mvnsite | 6m 40s | | the patch passed |
| -1 :x: | javadoc | 1m 23s |
[/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/26/artifact/out/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 18 new + 0
unchanged - 0 fixed = 18 total (was 0) |
| +1 :green_heart: | javadoc | 5m 47s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | spotbugs | 3m 4s |
[/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/26/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html)
| hadoop-common-project/hadoop-common generated 2 new + 0 unchanged - 0 fixed
= 2 total (was 0) |
| +1 :green_heart: | shadedclient | 24m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 38s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 54s | | hadoop-hdfs-client in the patch
[jira] [Work logged] (HDFS-16697) Randomly setting “dfs.namenode.resource.checked.volumes.minimum” will always prevent safe mode from being turned off
[
https://issues.apache.org/jira/browse/HDFS-16697?focusedWorklogId=796012=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796012
]
ASF GitHub Bot logged work on HDFS-16697:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 09:47
Start Date: 28/Jul/22 09:47
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4653:
URL: https://github.com/apache/hadoop/pull/4653#issuecomment-1197916607
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 28s | | trunk passed |
| +1 :green_heart: | compile | 1m 41s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 21s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 54s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 25s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/1/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch failed. |
| -1 :x: | compile | 0m 26s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| -1 :x: | javac | 0m 26s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| -1 :x: | compile | 0m 26s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
| -1 :x: | javac | 0m 26s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/1/artifact/out/blanks-eol.txt)
| The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 2s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4653/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 6 new + 8 unchanged -
0 fixed = 14 total (was 8) |
| -1 :x: | mvnsite | 0m 27s |
[jira] [Work logged] (HDFS-16619) Fix HttpHeaders.Values And HttpHeaders.Names Deprecated Import.
[
https://issues.apache.org/jira/browse/HDFS-16619?focusedWorklogId=795995=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795995
]
ASF GitHub Bot logged work on HDFS-16619:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 08:52
Start Date: 28/Jul/22 08:52
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4406:
URL: https://github.com/apache/hadoop/pull/4406#issuecomment-1197855725
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 21s | | trunk passed |
| +1 :green_heart: | compile | 1m 55s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 44s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 27m 10s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 35s | | the patch passed |
| +1 :green_heart: | compile | 1m 35s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 35s | |
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new +
911 unchanged - 26 fixed = 911 total (was 937) |
| +1 :green_heart: | compile | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 26s | |
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new
+ 890 unchanged - 26 fixed = 890 total (was 916) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 9s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 13s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 242m 56s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 363m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4406/16/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4406 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c6947778d078 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f516ecc01c880860201b5d2f1b5c80a96f90cdfd |
| Default Java | Private
[jira] [Work logged] (HDFS-16619) Fix HttpHeaders.Values And HttpHeaders.Names Deprecated Import.
[
https://issues.apache.org/jira/browse/HDFS-16619?focusedWorklogId=795992=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795992
]
ASF GitHub Bot logged work on HDFS-16619:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 08:46
Start Date: 28/Jul/22 08:46
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4406:
URL: https://github.com/apache/hadoop/pull/4406#issuecomment-1197849703
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 56s | | trunk passed |
| +1 :green_heart: | compile | 1m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 37s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 22s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 42s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 54s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 20s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 1m 33s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 33s | |
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new +
911 unchanged - 26 fixed = 911 total (was 937) |
| +1 :green_heart: | compile | 1m 24s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 24s | |
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new
+ 890 unchanged - 26 fixed = 890 total (was 916) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 3s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 1s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 248m 9s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 15s | | The patch does not
generate ASF License warnings. |
| | | 364m 17s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4406/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4406 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e3cc2e53831d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b1ce7d064b406ea7589546b245dfc78cc1f2e052 |
| Default Java | Private
[jira] [Commented] (HDFS-16660) Improve Code With Lambda in IPCLoggerChannel class
[ https://issues.apache.org/jira/browse/HDFS-16660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572313#comment-17572313 ] Steve Loughran commented on HDFS-16660: --- for anyone doing this, look in hadoop common package org.apache.hadoop.util.functional for function interfaces which can raise iOEs, helpers to unwrap exceptions, and support for using RemoteIterator for iterating over results. you need them if you are invoking our own APIs > Improve Code With Lambda in IPCLoggerChannel class > -- > > Key: HDFS-16660 > URL: https://issues.apache.org/jira/browse/HDFS-16660 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Improve Code With Lambda in IPCLoggerChannel class -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16697) Randomly setting “dfs.namenode.resource.checked.volumes.minimum” will always prevent safe mode from being turned off
[
https://issues.apache.org/jira/browse/HDFS-16697?focusedWorklogId=795980=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795980
]
ASF GitHub Bot logged work on HDFS-16697:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 08:13
Start Date: 28/Jul/22 08:13
Worklog Time Spent: 10m
Work Description: Likkey closed pull request #4649: HDFS-16697.Randomly
setting “dfs.namenode.resource.checked.volumes.minimum” will always prevent
safe mode from being turned off
URL: https://github.com/apache/hadoop/pull/4649
Issue Time Tracking
---
Worklog Id: (was: 795980)
Time Spent: 1.5h (was: 1h 20m)
> Randomly setting “dfs.namenode.resource.checked.volumes.minimum” will always
> prevent safe mode from being turned off
>
>
> Key: HDFS-16697
> URL: https://issues.apache.org/jira/browse/HDFS-16697
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.3
> Environment: Linux version 4.15.0-142-generic
> (buildd@lgw01-amd64-039) (gcc version 5.4.0 20160609 (Ubuntu
> 5.4.0-6ubuntu1~16.04.12))
> java version "1.8.0_162"
> Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
> Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
>Reporter: Jingxuan Fu
>Assignee: Jingxuan Fu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.3
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> {code:java}
>
> dfs.namenode.resource.checked.volumes.minimum
> 1
>
> The minimum number of redundant NameNode storage volumes required.
>
> {code}
> I found that when setting the value of
> “dfs.namenode.resource.checked.volumes.minimum” is greater than the total
> number of storage volumes in the NameNode, it is always impossible to turn
> off the safe mode, and when in safe mode, the file system only accepts read
> data requests, but not delete, modify and other change requests, which is
> greatly limited by the function.
> The default value of the configuration item is 1, we set to 2 as an example
> for illustration, after starting hdfs logs and the client will throw the
> relevant reminders.
> {code:java}
> 2022-07-27 17:37:31,772 WARN
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: NameNode low on
> available disk space. Already in safe mode.
> 2022-07-27 17:37:31,772 INFO org.apache.hadoop.hdfs.StateChange: STATE* Safe
> mode is ON.
> Resources are low on NN. Please add or free up more resourcesthen turn off
> safe mode manually. NOTE: If you turn off safe mode before adding resources,
> the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode
> leave" to turn safe mode off.
> {code}
> {code:java}
> org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create
> directory /hdfsapi/test. Name node is in safe mode.
> Resources are low on NN. Please add or free up more resourcesthen turn off
> safe mode manually. NOTE: If you turn off safe mode before adding resources,
> the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode
> leave" to turn safe mode off. NamenodeHostName:192.168.1.167
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.newSafemodeException(FSNamesystem.java:1468)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1455)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3174)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1145)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:714)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:527)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1036)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1000)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:928)
> at java.base/java.security.AccessController.doPrivileged(Native
> Method)
> at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2916){code}
> According to the prompt, it is believed that there is not enough resource
> space to meet the corresponding
[jira] [Work logged] (HDFS-16697) Randomly setting “dfs.namenode.resource.checked.volumes.minimum” will always prevent safe mode from being turned off
[
https://issues.apache.org/jira/browse/HDFS-16697?focusedWorklogId=795979=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795979
]
ASF GitHub Bot logged work on HDFS-16697:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 08:13
Start Date: 28/Jul/22 08:13
Worklog Time Spent: 10m
Work Description: Likkey opened a new pull request, #4653:
URL: https://github.com/apache/hadoop/pull/4653
### Description of PR
Add Precondition.checkArgument() for minimumRedundantVolumes to ensure that
the value is greater than the number of NameNode storage volumes to avoid never
being able to turn off safe mode afterwards.
JIRA:[[HDFS-16697](https://issues.apache.org/jira/browse/HDFS-16697)]
### How was this patch tested?
It is found that “dfs.namenode.resource.checked.volumes.minimum” lacks a
condition check and an associated exception handling mechanism, which makes it
impossible to find the root cause of the impact when a misconfiguration occurs.
This patch provides a check of the configuration items，it will throw a
IllegalArgumentException and detailed error message when the value of
"dfs.namenode.resource.checked.volumes.minimum" is set greater than the number
of NameNode storage volumes to avoid the misconfiguration from affecting the
subsequent operation of the program.
Issue Time Tracking
---
Worklog Id: (was: 795979)
Time Spent: 1h 20m (was: 1h 10m)
> Randomly setting “dfs.namenode.resource.checked.volumes.minimum” will always
> prevent safe mode from being turned off
>
>
> Key: HDFS-16697
> URL: https://issues.apache.org/jira/browse/HDFS-16697
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.3
> Environment: Linux version 4.15.0-142-generic
> (buildd@lgw01-amd64-039) (gcc version 5.4.0 20160609 (Ubuntu
> 5.4.0-6ubuntu1~16.04.12))
> java version "1.8.0_162"
> Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
> Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
>Reporter: Jingxuan Fu
>Assignee: Jingxuan Fu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.3
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> {code:java}
>
> dfs.namenode.resource.checked.volumes.minimum
> 1
>
> The minimum number of redundant NameNode storage volumes required.
>
> {code}
> I found that when setting the value of
> “dfs.namenode.resource.checked.volumes.minimum” is greater than the total
> number of storage volumes in the NameNode, it is always impossible to turn
> off the safe mode, and when in safe mode, the file system only accepts read
> data requests, but not delete, modify and other change requests, which is
> greatly limited by the function.
> The default value of the configuration item is 1, we set to 2 as an example
> for illustration, after starting hdfs logs and the client will throw the
> relevant reminders.
> {code:java}
> 2022-07-27 17:37:31,772 WARN
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: NameNode low on
> available disk space. Already in safe mode.
> 2022-07-27 17:37:31,772 INFO org.apache.hadoop.hdfs.StateChange: STATE* Safe
> mode is ON.
> Resources are low on NN. Please add or free up more resourcesthen turn off
> safe mode manually. NOTE: If you turn off safe mode before adding resources,
> the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode
> leave" to turn safe mode off.
> {code}
> {code:java}
> org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create
> directory /hdfsapi/test. Name node is in safe mode.
> Resources are low on NN. Please add or free up more resourcesthen turn off
> safe mode manually. NOTE: If you turn off safe mode before adding resources,
> the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode
> leave" to turn safe mode off. NamenodeHostName:192.168.1.167
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.newSafemodeException(FSNamesystem.java:1468)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1455)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3174)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1145)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:714)
> at
>
[jira] [Work logged] (HDFS-16619) Fix HttpHeaders.Values And HttpHeaders.Names Deprecated Import.
[
https://issues.apache.org/jira/browse/HDFS-16619?focusedWorklogId=795977=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795977
]
ASF GitHub Bot logged work on HDFS-16619:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 08:08
Start Date: 28/Jul/22 08:08
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4406:
URL: https://github.com/apache/hadoop/pull/4406#issuecomment-1197808522
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 7s | | trunk passed |
| +1 :green_heart: | compile | 1m 46s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 21s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 43s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 21s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 45s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 23s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 23s | |
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new +
911 unchanged - 26 fixed = 911 total (was 937) |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 19s | |
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new
+ 890 unchanged - 26 fixed = 890 total (was 916) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 59s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 250m 33s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 361m 53s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4406/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4406 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 2d5a2a65e261 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e7a0588348cb8da5501d7528a810a7ccf6c94963 |
| Default Java | Private
[jira] [Work logged] (HDFS-16671) RBF: RouterRpcFairnessPolicyController supports configurable permit acquire timeout
[
https://issues.apache.org/jira/browse/HDFS-16671?focusedWorklogId=795969=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795969
]
ASF GitHub Bot logged work on HDFS-16671:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 07:44
Start Date: 28/Jul/22 07:44
Worklog Time Spent: 10m
Work Description: ferhui commented on PR #4597:
URL: https://github.com/apache/hadoop/pull/4597#issuecomment-1197785576
Merged! @ZanderXu Thanks for your contribution. @goiri @ayushtkn @slfan1989
Thanks for your reviews!
Issue Time Tracking
---
Worklog Id: (was: 795969)
Time Spent: 3h (was: 2h 50m)
> RBF: RouterRpcFairnessPolicyController supports configurable permit acquire
> timeout
> ---
>
> Key: HDFS-16671
> URL: https://issues.apache.org/jira/browse/HDFS-16671
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.3.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> RouterRpcFairnessPolicyController supports configurable permit acquire
> timeout. Hardcode 1s is very long, and it has causes an incident in our prod
> environment when one nameserivce is busy.
> And the optimal timeout maybe should be less than p50(avgTime).
> And all handlers in RBF is waiting to acquire the permit of the busy ns.
> {code:java}
> "IPC Server handler 12 on default port " #2370 daemon prio=5 os_prio=0
> tid=? nid=? waiting on condition [?]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for (a
> java.util.concurrent.Semaphore$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:409)
> at
> org.apache.hadoop.hdfs.server.federation.fairness.AbstractRouterRpcFairnessPolicyController.acquirePermit(AbstractRouterRpcFairnessPolicyController.java:56)
> at
> org.apache.hadoop.hdfs.server.federation.fairness.DynamicRouterRpcFairnessPolicyController.acquirePermit(DynamicRouterRpcFairnessPolicyController.java:123)
> at
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.acquirePermit(RouterRpcClient.java:1500)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-16671) RBF: RouterRpcFairnessPolicyController supports configurable permit acquire timeout
[
https://issues.apache.org/jira/browse/HDFS-16671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hui Fei resolved HDFS-16671.
Fix Version/s: 1.3.0
Resolution: Fixed
> RBF: RouterRpcFairnessPolicyController supports configurable permit acquire
> timeout
> ---
>
> Key: HDFS-16671
> URL: https://issues.apache.org/jira/browse/HDFS-16671
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.3.0
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> RouterRpcFairnessPolicyController supports configurable permit acquire
> timeout. Hardcode 1s is very long, and it has causes an incident in our prod
> environment when one nameserivce is busy.
> And the optimal timeout maybe should be less than p50(avgTime).
> And all handlers in RBF is waiting to acquire the permit of the busy ns.
> {code:java}
> "IPC Server handler 12 on default port " #2370 daemon prio=5 os_prio=0
> tid=? nid=? waiting on condition [?]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for (a
> java.util.concurrent.Semaphore$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:409)
> at
> org.apache.hadoop.hdfs.server.federation.fairness.AbstractRouterRpcFairnessPolicyController.acquirePermit(AbstractRouterRpcFairnessPolicyController.java:56)
> at
> org.apache.hadoop.hdfs.server.federation.fairness.DynamicRouterRpcFairnessPolicyController.acquirePermit(DynamicRouterRpcFairnessPolicyController.java:123)
> at
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.acquirePermit(RouterRpcClient.java:1500)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16671) RBF: RouterRpcFairnessPolicyController supports configurable permit acquire timeout
[
https://issues.apache.org/jira/browse/HDFS-16671?focusedWorklogId=795967=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795967
]
ASF GitHub Bot logged work on HDFS-16671:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 07:42
Start Date: 28/Jul/22 07:42
Worklog Time Spent: 10m
Work Description: ferhui merged PR #4597:
URL: https://github.com/apache/hadoop/pull/4597
Issue Time Tracking
---
Worklog Id: (was: 795967)
Time Spent: 2h 50m (was: 2h 40m)
> RBF: RouterRpcFairnessPolicyController supports configurable permit acquire
> timeout
> ---
>
> Key: HDFS-16671
> URL: https://issues.apache.org/jira/browse/HDFS-16671
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> RouterRpcFairnessPolicyController supports configurable permit acquire
> timeout. Hardcode 1s is very long, and it has causes an incident in our prod
> environment when one nameserivce is busy.
> And the optimal timeout maybe should be less than p50(avgTime).
> And all handlers in RBF is waiting to acquire the permit of the busy ns.
> {code:java}
> "IPC Server handler 12 on default port " #2370 daemon prio=5 os_prio=0
> tid=? nid=? waiting on condition [?]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for (a
> java.util.concurrent.Semaphore$NonfairSync)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:409)
> at
> org.apache.hadoop.hdfs.server.federation.fairness.AbstractRouterRpcFairnessPolicyController.acquirePermit(AbstractRouterRpcFairnessPolicyController.java:56)
> at
> org.apache.hadoop.hdfs.server.federation.fairness.DynamicRouterRpcFairnessPolicyController.acquirePermit(DynamicRouterRpcFairnessPolicyController.java:123)
> at
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.acquirePermit(RouterRpcClient.java:1500)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-16658) BlockManager should output some logs when logEveryBlock is true.
[
https://issues.apache.org/jira/browse/HDFS-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoqiao He resolved HDFS-16658.
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
Committed to trunk. Thanks [~xuzq_zander].
> BlockManager should output some logs when logEveryBlock is true.
>
>
> Key: HDFS-16658
> URL: https://issues.apache.org/jira/browse/HDFS-16658
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> During locating some abnormal cases in our prod environment, I found that
> BlockManager does not out put some logs in `addStoredBlock` even though
> `logEveryBlock` is true.
> I feel that we need to change the log level from DEBUG to INFO.
> {code:java}
> // Some comments here
> private Block addStoredBlock(final BlockInfo block,
>final Block reportedBlock,
>DatanodeStorageInfo storageInfo,
>DatanodeDescriptor delNodeHint,
>boolean logEveryBlock)
> throws IOException {
>
> if (logEveryBlock) {
> blockLog.debug("BLOCK* addStoredBlock: {} is added to {} (size={})",
> node, storedBlock, storedBlock.getNumBytes());
> }
> ...
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16697) Randomly setting “dfs.namenode.resource.checked.volumes.minimum” will always prevent safe mode from being turned off
[
https://issues.apache.org/jira/browse/HDFS-16697?focusedWorklogId=795963=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795963
]
ASF GitHub Bot logged work on HDFS-16697:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 07:31
Start Date: 28/Jul/22 07:31
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4649:
URL: https://github.com/apache/hadoop/pull/4649#issuecomment-1197772721
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 52s | | trunk passed |
| +1 :green_heart: | compile | 1m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 20s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 8s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 51s |
[/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4649/1/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch failed. |
| -1 :x: | compile | 0m 55s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4649/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| -1 :x: | javac | 0m 55s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4649/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| -1 :x: | compile | 0m 50s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4649/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
| -1 :x: | javac | 0m 50s |
[/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4649/1/artifact/out/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4649/1/artifact/out/blanks-eol.txt)
| The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 1s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4649/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 2 unchanged -
0 fixed = 6 total (was 2) |
| -1 :x: | mvnsite |
[jira] [Work logged] (HDFS-16658) BlockManager should output some logs when logEveryBlock is true.
[
https://issues.apache.org/jira/browse/HDFS-16658?focusedWorklogId=795962=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795962
]
ASF GitHub Bot logged work on HDFS-16658:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 07:31
Start Date: 28/Jul/22 07:31
Worklog Time Spent: 10m
Work Description: Hexiaoqiao commented on PR #4559:
URL: https://github.com/apache/hadoop/pull/4559#issuecomment-1197772469
Committed to trunk. Thanks @ZanderXu for your contributions. Thanks
@slfan1989 for your comments.
Issue Time Tracking
---
Worklog Id: (was: 795962)
Time Spent: 1.5h (was: 1h 20m)
> BlockManager should output some logs when logEveryBlock is true.
>
>
> Key: HDFS-16658
> URL: https://issues.apache.org/jira/browse/HDFS-16658
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> During locating some abnormal cases in our prod environment, I found that
> BlockManager does not out put some logs in `addStoredBlock` even though
> `logEveryBlock` is true.
> I feel that we need to change the log level from DEBUG to INFO.
> {code:java}
> // Some comments here
> private Block addStoredBlock(final BlockInfo block,
>final Block reportedBlock,
>DatanodeStorageInfo storageInfo,
>DatanodeDescriptor delNodeHint,
>boolean logEveryBlock)
> throws IOException {
>
> if (logEveryBlock) {
> blockLog.debug("BLOCK* addStoredBlock: {} is added to {} (size={})",
> node, storedBlock, storedBlock.getNumBytes());
> }
> ...
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16658) BlockManager should output some logs when logEveryBlock is true.
[
https://issues.apache.org/jira/browse/HDFS-16658?focusedWorklogId=795961=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795961
]
ASF GitHub Bot logged work on HDFS-16658:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 07:30
Start Date: 28/Jul/22 07:30
Worklog Time Spent: 10m
Work Description: Hexiaoqiao merged PR #4559:
URL: https://github.com/apache/hadoop/pull/4559
Issue Time Tracking
---
Worklog Id: (was: 795961)
Time Spent: 1h 20m (was: 1h 10m)
> BlockManager should output some logs when logEveryBlock is true.
>
>
> Key: HDFS-16658
> URL: https://issues.apache.org/jira/browse/HDFS-16658
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> During locating some abnormal cases in our prod environment, I found that
> BlockManager does not out put some logs in `addStoredBlock` even though
> `logEveryBlock` is true.
> I feel that we need to change the log level from DEBUG to INFO.
> {code:java}
> // Some comments here
> private Block addStoredBlock(final BlockInfo block,
>final Block reportedBlock,
>DatanodeStorageInfo storageInfo,
>DatanodeDescriptor delNodeHint,
>boolean logEveryBlock)
> throws IOException {
>
> if (logEveryBlock) {
> blockLog.debug("BLOCK* addStoredBlock: {} is added to {} (size={})",
> node, storedBlock, storedBlock.getNumBytes());
> }
> ...
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-13522) RBF: Support observer node from Router-Based Federation
[
https://issues.apache.org/jira/browse/HDFS-13522?focusedWorklogId=795959=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795959
]
ASF GitHub Bot logged work on HDFS-13522:
-
Author: ASF GitHub Bot
Created on: 28/Jul/22 07:25
Start Date: 28/Jul/22 07:25
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4311:
URL: https://github.com/apache/hadoop/pull/4311#issuecomment-1197766744
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 3s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 26s | | trunk passed |
| +1 :green_heart: | compile | 26m 7s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 22m 59s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 57s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 18s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 6m 10s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 12m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 28s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 48s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 32s | | the patch passed |
| +1 :green_heart: | compile | 25m 6s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 25m 6s | | the patch passed |
| +1 :green_heart: | javac | 25m 6s | | the patch passed |
| +1 :green_heart: | compile | 21m 36s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 21m 36s | | the patch passed |
| +1 :green_heart: | javac | 21m 36s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 31s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4311/18/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 177 unchanged - 1 fixed = 181 total (was
178) |
| +1 :green_heart: | mvnsite | 6m 59s | | the patch passed |
| -1 :x: | javadoc | 1m 31s |
[/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4311/18/artifact/out/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 18 new + 0
unchanged - 0 fixed = 18 total (was 0) |
| +1 :green_heart: | javadoc | 6m 54s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | spotbugs | 3m 14s |
[/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4311/18/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html)
| hadoop-common-project/hadoop-common generated 2 new + 0 unchanged - 0 fixed
= 2 total (was 0) |
| +1 :green_heart: | shadedclient | 22m 37s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 40s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 18s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 246m 3s | | hadoop-hdfs in
