[jira] [Comment Edited] (HDFS-12643) HDFS maintenance state behaviour is confusing and not well documented
[
https://issues.apache.org/jira/browse/HDFS-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421255#comment-17421255
]
TisNotT edited comment on HDFS-12643 at 10/11/21, 3:46 AM:
---
it is confuse to me too.So I am finding whether there is a way to set special
datanodes to maintenance state or not. If not, maybe I will dev one for my
company.
was (Author: sharpshow):
it is confuse to me too.So I am finding whether there is an api to set special
datanodes to maintenance state or not. If not,maybe I will dev one for my
company.
> HDFS maintenance state behaviour is confusing and not well documented
> -
>
> Key: HDFS-12643
> URL: https://issues.apache.org/jira/browse/HDFS-12643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation, namenode
>Reporter: Andre Araujo
>Priority: Major
>
> The current implementation of the HDFS maintenance state feature is confusing
> and error-prone. The documentation is missing important information that's
> required for the correct use of the feature.
> For example, if the Hadoop admin wants to put a single node in maintenance
> state, he/she can add a single entry to the maintenance file with the
> contents:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "IN_MAINTENANCE",
>"maintenanceExpireTimeInMS": 1507663698000
> }
> {code}
> Let's say now that the actual maintenance finished well before the set
> expiration time and the Hadoop admin wants to bring the node back to NORMAL
> state. It would be natural to simply change the state of the node, as show
> below, and run another refresh:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "NORMAL"
> }
> {code}
> The configuration file above, though, not only take the node {{host-1}} out
> of maintenance state but it also *blacklists all the other DataNodes*. This
> behaviour seems inconsistent to me and is due to {{emptyInServiceNodeLists}}
> being set to {{false}}
> [here|https://github.com/apache/hadoop/blob/230b85d5865b7e08fb7aaeab45295b5b966011ef/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java#L80]
> only when there is at least one node with {{adminState = NORMAL}} listed in
> the file.
> I believe that it would be more consistent, and less error prone, to simply
> implement the following:
> * If the dfs.hosts file is empty, all nodes are allowed and in normal state
> * If the file is not empty, any host *not* listed in the file is
> *blacklisted*, regardless of the state of the hosts listed in the file.
> Regardless of the implementation being changed or not, the documentation also
> needs to be updated to ensure the readers know of the caveats mentioned above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16266) Add remote port information to HDFS audit log
[
https://issues.apache.org/jira/browse/HDFS-16266?focusedWorklogId=663292=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663292
]
ASF GitHub Bot logged work on HDFS-16266:
-
Author: ASF GitHub Bot
Created on: 11/Oct/21 00:14
Start Date: 11/Oct/21 00:14
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3538:
URL: https://github.com/apache/hadoop/pull/3538#issuecomment-939580772
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 53s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 54s | | trunk passed |
| +1 :green_heart: | compile | 21m 25s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 47s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 14s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 20s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 25s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 8s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 8s | | the patch passed |
| +1 :green_heart: | compile | 20m 39s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 39s | | the patch passed |
| +1 :green_heart: | compile | 18m 33s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 36s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3538/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 485 unchanged - 2 fixed = 487 total (was
487) |
| +1 :green_heart: | mvnsite | 3m 13s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 19s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 12s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 27s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 373m 24s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3538/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not
generate ASF License warnings. |
| | | 589m 50s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.fs.viewfs.TestViewFSOverloadSchemeWithMountTableConfigInHDFS |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.namenode.TestAuditLogger |
| |
hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeHdfsFileSystemContract |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3538/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3538 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 65cac33ec84e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC
[jira] [Work logged] (HDFS-16266) Add remote port information to HDFS audit log
[
https://issues.apache.org/jira/browse/HDFS-16266?focusedWorklogId=663290=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663290
]
ASF GitHub Bot logged work on HDFS-16266:
-
Author: ASF GitHub Bot
Created on: 11/Oct/21 00:03
Start Date: 11/Oct/21 00:03
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #3538:
URL: https://github.com/apache/hadoop/pull/3538#discussion_r725721616
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
##
@@ -409,9 +420,9 @@ public static String getAuxiliaryPortEstablishedQOP() {
Call call = CurCall.get();
return call != null ? call.clientId : RpcConstants.DUMMY_CLIENT_ID;
}
-
+
/** Returns remote address as a string when invoked inside an RPC.
- * Returns null in case of an error.
+ * Returns null in case of an error.n
Review comment:
> Extra `n`
Thanks for your review. I fixed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 663290)
Time Spent: 0.5h (was: 20m)
> Add remote port information to HDFS audit log
> -
>
> Key: HDFS-16266
> URL: https://issues.apache.org/jira/browse/HDFS-16266
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In our production environment, we occasionally encounter a problem where a
> user submits an abnormal computation task, causing a sudden flood of
> requests, which causes the queueTime and processingTime of the Namenode to
> rise very high, causing a large backlog of tasks.
> We usually locate and kill specific Spark, Flink, or MapReduce tasks based on
> metrics and audit logs. Currently, IP and UGI are recorded in audit logs, but
> there is no port information, so it is difficult to locate specific processes
> sometimes. Therefore, I propose that we add the port information to the audit
> log, so that we can easily track the upstream process.
> Currently, some projects contain port information in audit logs, such as
> Hbase and Alluxio. I think it is also necessary to add port information for
> HDFS audit logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16262) Async refresh of cached locations in DFSInputStream
[ https://issues.apache.org/jira/browse/HDFS-16262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426826#comment-17426826 ] Ahmed Hussein commented on HDFS-16262: -- Thanks [~bbeaudreault] for working on the asynchronous implementation of HDFS-15119 This was on my Todo for quite sometime. Hopefully, I will get time to take look at the PR soon. > Async refresh of cached locations in DFSInputStream > --- > > Key: HDFS-16262 > URL: https://issues.apache.org/jira/browse/HDFS-16262 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > HDFS-15119 added the ability to invalidate cached block locations in > DFSInputStream. As written, the feature will affect all DFSInputStreams > regardless of whether they need it or not. The invalidation also only applies > on the next request, so the next request will pay the cost of calling > openInfo before reading the data. > I'm working on a feature for HBase which enables efficient healing of > locality through Balancer-style low level block moves (HBASE-26250). I'd like > to utilize the idea started in HDFS-15119 in order to update DFSInputStreams > after blocks have been moved to local hosts. > I was considering using the feature as is, but some of our clusters are quite > large and I'm concerned about the impact on the namenode: > * We have some clusters with over 350k StoreFiles, so that'd be 350k > DFSInputStreams. With such a large number and very active usage, having the > refresh be in-line makes it too hard to ensure we don't DDOS the NameNode. > * Currently we need to pay the price of openInfo the next time a > DFSInputStream is invoked. Moving that async would minimize the latency hit. > Also, some StoreFiles might be far less frequently accessed, so they may live > on for a long time before ever refreshing. We'd like to be able to know that > all DFSInputStreams are refreshed by a given time. > * We may have 350k files, but only a small percentage of them are ever > non-local at a given time. Refreshing only if necessary will save a lot of > work. > In order to make this as painless to end users as possible, I'd like to: > * Update the implementation to utilize an async thread for managing > refreshes. This will give more control over rate limiting across all > DFSInputStreams in a DFSClient, and also ensure that all DFSInputStreams are > refreshed. > * Only refresh files which are lacking a local replica or have known > deadNodes to be cleaned up > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16266) Add remote port information to HDFS audit log
[
https://issues.apache.org/jira/browse/HDFS-16266?focusedWorklogId=663259=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663259
]
ASF GitHub Bot logged work on HDFS-16266:
-
Author: ASF GitHub Bot
Created on: 10/Oct/21 15:10
Start Date: 10/Oct/21 15:10
Worklog Time Spent: 10m
Work Description: yakirgb commented on a change in pull request #3538:
URL: https://github.com/apache/hadoop/pull/3538#discussion_r725653854
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
##
@@ -409,9 +420,9 @@ public static String getAuxiliaryPortEstablishedQOP() {
Call call = CurCall.get();
return call != null ? call.clientId : RpcConstants.DUMMY_CLIENT_ID;
}
-
+
/** Returns remote address as a string when invoked inside an RPC.
- * Returns null in case of an error.
+ * Returns null in case of an error.n
Review comment:
Extra `n`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 663259)
Time Spent: 20m (was: 10m)
> Add remote port information to HDFS audit log
> -
>
> Key: HDFS-16266
> URL: https://issues.apache.org/jira/browse/HDFS-16266
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In our production environment, we occasionally encounter a problem where a
> user submits an abnormal computation task, causing a sudden flood of
> requests, which causes the queueTime and processingTime of the Namenode to
> rise very high, causing a large backlog of tasks.
> We usually locate and kill specific Spark, Flink, or MapReduce tasks based on
> metrics and audit logs. Currently, IP and UGI are recorded in audit logs, but
> there is no port information, so it is difficult to locate specific processes
> sometimes. Therefore, I propose that we add the port information to the audit
> log, so that we can easily track the upstream process.
> Currently, some projects contain port information in audit logs, such as
> Hbase and Alluxio. I think it is also necessary to add port information for
> HDFS audit logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16260) Make hdfs_deleteSnapshot tool cross platform
[
https://issues.apache.org/jira/browse/HDFS-16260?focusedWorklogId=663256=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663256
]
ASF GitHub Bot logged work on HDFS-16260:
-
Author: ASF GitHub Bot
Created on: 10/Oct/21 14:37
Start Date: 10/Oct/21 14:37
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3532:
URL: https://github.com/apache/hadoop/pull/3532#issuecomment-939493466
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 20m 27s | | trunk passed |
| +1 :green_heart: | compile | 2m 50s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 2m 50s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | mvnsite | 0m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 45m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 18s | | the patch passed |
| +1 :green_heart: | compile | 2m 40s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | cc | 2m 40s | | the patch passed |
| +1 :green_heart: | golang | 2m 40s | | the patch passed |
| +1 :green_heart: | javac | 2m 40s | | the patch passed |
| +1 :green_heart: | compile | 2m 45s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | cc | 2m 45s | | the patch passed |
| +1 :green_heart: | golang | 2m 45s | | the patch passed |
| +1 :green_heart: | javac | 2m 45s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 20s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 10s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 32m 13s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 105m 17s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3532 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux d66501f5124f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 27a054227092f6574759c5b9c312f38f4bb0b788 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/testReport/ |
| Max. process+thread count | 719 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 663256)
Time Spent: 1.5h (was: 1h 20m)
> Make hdfs_deleteSnapshot tool cross platform
>
[jira] [Updated] (HDFS-16266) Add remote port information to HDFS audit log
[ https://issues.apache.org/jira/browse/HDFS-16266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-16266: -- Labels: pull-request-available (was: ) > Add remote port information to HDFS audit log > - > > Key: HDFS-16266 > URL: https://issues.apache.org/jira/browse/HDFS-16266 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > In our production environment, we occasionally encounter a problem where a > user submits an abnormal computation task, causing a sudden flood of > requests, which causes the queueTime and processingTime of the Namenode to > rise very high, causing a large backlog of tasks. > We usually locate and kill specific Spark, Flink, or MapReduce tasks based on > metrics and audit logs. Currently, IP and UGI are recorded in audit logs, but > there is no port information, so it is difficult to locate specific processes > sometimes. Therefore, I propose that we add the port information to the audit > log, so that we can easily track the upstream process. > Currently, some projects contain port information in audit logs, such as > Hbase and Alluxio. I think it is also necessary to add port information for > HDFS audit logs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16266) Add remote port information to HDFS audit log
[ https://issues.apache.org/jira/browse/HDFS-16266?focusedWorklogId=663255=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663255 ] ASF GitHub Bot logged work on HDFS-16266: - Author: ASF GitHub Bot Created on: 10/Oct/21 14:23 Start Date: 10/Oct/21 14:23 Worklog Time Spent: 10m Work Description: tomscut opened a new pull request #3538: URL: https://github.com/apache/hadoop/pull/3538 JIRA: [HDFS-16266](https://issues.apache.org/jira/browse/HDFS-16266) In our production environment, we occasionally encounter a problem where a user submits an abnormal computation task, causing a sudden flood of requests, which causes the queueTime and processingTime of the Namenode to rise very high, causing a large backlog of tasks. We usually locate and kill specific Spark, Flink, or MapReduce tasks based on metrics and audit logs. Currently, IP and UGI are recorded in audit logs, but there is no port information, so it is difficult to locate specific processes sometimes. Therefore, I propose that we add the port information to the audit log, so that we can easily track the upstream process. Currently, some projects contain port information in audit logs, such as Hbase and Alluxio. I think it is also necessary to add port information for HDFS audit logs. Before:  After:  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 663255) Remaining Estimate: 0h Time Spent: 10m > Add remote port information to HDFS audit log > - > > Key: HDFS-16266 > URL: https://issues.apache.org/jira/browse/HDFS-16266 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: tomscut >Assignee: tomscut >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > In our production environment, we occasionally encounter a problem where a > user submits an abnormal computation task, causing a sudden flood of > requests, which causes the queueTime and processingTime of the Namenode to > rise very high, causing a large backlog of tasks. > We usually locate and kill specific Spark, Flink, or MapReduce tasks based on > metrics and audit logs. Currently, IP and UGI are recorded in audit logs, but > there is no port information, so it is difficult to locate specific processes > sometimes. Therefore, I propose that we add the port information to the audit > log, so that we can easily track the upstream process. > Currently, some projects contain port information in audit logs, such as > Hbase and Alluxio. I think it is also necessary to add port information for > HDFS audit logs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16266) Add remote port information to HDFS audit log
tomscut created HDFS-16266: -- Summary: Add remote port information to HDFS audit log Key: HDFS-16266 URL: https://issues.apache.org/jira/browse/HDFS-16266 Project: Hadoop HDFS Issue Type: Improvement Reporter: tomscut Assignee: tomscut In our production environment, we occasionally encounter a problem where a user submits an abnormal computation task, causing a sudden flood of requests, which causes the queueTime and processingTime of the Namenode to rise very high, causing a large backlog of tasks. We usually locate and kill specific Spark, Flink, or MapReduce tasks based on metrics and audit logs. Currently, IP and UGI are recorded in audit logs, but there is no port information, so it is difficult to locate specific processes sometimes. Therefore, I propose that we add the port information to the audit log, so that we can easily track the upstream process. Currently, some projects contain port information in audit logs, such as Hbase and Alluxio. I think it is also necessary to add port information for HDFS audit logs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16260) Make hdfs_deleteSnapshot tool cross platform
[
https://issues.apache.org/jira/browse/HDFS-16260?focusedWorklogId=663251=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663251
]
ASF GitHub Bot logged work on HDFS-16260:
-
Author: ASF GitHub Bot
Created on: 10/Oct/21 12:52
Start Date: 10/Oct/21 12:52
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3532:
URL: https://github.com/apache/hadoop/pull/3532#issuecomment-939477215
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 11m 8s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 25m 3s | | trunk passed |
| +1 :green_heart: | compile | 2m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 52m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 16s | | the patch passed |
| +1 :green_heart: | compile | 2m 26s | | the patch passed |
| +1 :green_heart: | cc | 2m 26s | | the patch passed |
| +1 :green_heart: | golang | 2m 26s | | the patch passed |
| +1 :green_heart: | javac | 2m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 19s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 31s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 31m 41s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 125m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3532 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux 431de59b1c4f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 27a054227092f6574759c5b9c312f38f4bb0b788 |
| Default Java | Debian-11.0.12+7-post-Debian-2deb10u1 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/testReport/ |
| Max. process+thread count | 571 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/console |
| versions | git=2.20.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 663251)
Time Spent: 1h 20m (was: 1h 10m)
> Make hdfs_deleteSnapshot tool cross platform
>
>
> Key: HDFS-16260
> URL: https://issues.apache.org/jira/browse/HDFS-16260
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client, libhdfs++, tools
>Affects Versions: 3.4.0
> Environment: Centos 7, Centos 8, Debian 10, Ubuntu Focal
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The source files for hdfs_deleteSnapshot uses *getopt* for parsing the
> command line arguments. getopt is available only on Linux and thus, isn't
> cross platform. We need to replace getopt
[jira] [Work logged] (HDFS-16260) Make hdfs_deleteSnapshot tool cross platform
[
https://issues.apache.org/jira/browse/HDFS-16260?focusedWorklogId=663247=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663247
]
ASF GitHub Bot logged work on HDFS-16260:
-
Author: ASF GitHub Bot
Created on: 10/Oct/21 10:46
Start Date: 10/Oct/21 10:46
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3532:
URL: https://github.com/apache/hadoop/pull/3532#issuecomment-939457389
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 21m 9s | | trunk passed |
| +1 :green_heart: | compile | 2m 55s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 43m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 2m 39s | | the patch passed |
| +1 :green_heart: | cc | 2m 39s | | the patch passed |
| +1 :green_heart: | golang | 2m 39s | | the patch passed |
| +1 :green_heart: | javac | 2m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 59s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 32m 19s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 47s | | The patch does not
generate ASF License warnings. |
| | | 100m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3532 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux d99b19640345 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 27a054227092f6574759c5b9c312f38f4bb0b788 |
| Default Java | Red Hat, Inc.-1.8.0_302-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/testReport/ |
| Max. process+thread count | 719 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/console |
| versions | git=2.27.0 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 663247)
Time Spent: 1h 10m (was: 1h)
> Make hdfs_deleteSnapshot tool cross platform
>
>
> Key: HDFS-16260
> URL: https://issues.apache.org/jira/browse/HDFS-16260
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client, libhdfs++, tools
>Affects Versions: 3.4.0
> Environment: Centos 7, Centos 8, Debian 10, Ubuntu Focal
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The source files for hdfs_deleteSnapshot uses *getopt* for parsing the
> command line arguments. getopt is available only on Linux and thus, isn't
> cross platform. We need to replace getopt
[jira] [Work logged] (HDFS-16260) Make hdfs_deleteSnapshot tool cross platform
[
https://issues.apache.org/jira/browse/HDFS-16260?focusedWorklogId=663239=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663239
]
ASF GitHub Bot logged work on HDFS-16260:
-
Author: ASF GitHub Bot
Created on: 10/Oct/21 09:05
Start Date: 10/Oct/21 09:05
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3532:
URL: https://github.com/apache/hadoop/pull/3532#issuecomment-939435361
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 42s | | trunk passed |
| +1 :green_heart: | compile | 2m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 53m 40s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 16s | | the patch passed |
| +1 :green_heart: | compile | 2m 33s | | the patch passed |
| +1 :green_heart: | cc | 2m 33s | | the patch passed |
| +1 :green_heart: | golang | 2m 33s | | the patch passed |
| +1 :green_heart: | javac | 2m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 32m 0s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 109m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3532 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux 5b6c629aa6f0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 27a054227092f6574759c5b9c312f38f4bb0b788 |
| Default Java | Red Hat, Inc.-1.8.0_302-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/testReport/ |
| Max. process+thread count | 724 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3532/2/console |
| versions | git=2.9.5 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 663239)
Time Spent: 1h (was: 50m)
> Make hdfs_deleteSnapshot tool cross platform
>
>
> Key: HDFS-16260
> URL: https://issues.apache.org/jira/browse/HDFS-16260
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client, libhdfs++, tools
>Affects Versions: 3.4.0
> Environment: Centos 7, Centos 8, Debian 10, Ubuntu Focal
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The source files for hdfs_deleteSnapshot uses *getopt* for parsing the
> command line arguments. getopt is available only on Linux and thus, isn't
> cross platform. We need to replace getopt with
[jira] [Updated] (HDFS-13514) BenchmarkThroughput.readLocalFile hangs with misconfigured BUFFER_SIZE
[
https://issues.apache.org/jira/browse/HDFS-13514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDFS-13514:
--
Labels: pull-request-available (was: )
> BenchmarkThroughput.readLocalFile hangs with misconfigured BUFFER_SIZE
> --
>
> Key: HDFS-13514
> URL: https://issues.apache.org/jira/browse/HDFS-13514
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Affects Versions: 2.5.0
>Reporter: John Doe
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When the BUFFER_SIZE is configured to be 0, the while loop in
> BenchmarkThroughput.readLocalFile function hangs endlessly.
> This is because when the data.size (i.e., BUFFER_SIZE) is 0, the val will
> always be 0 by invoking val=in.read(data).
> Here is the code snippet.
> {code:java}
> BUFFER_SIZE = conf.getInt("dfsthroughput.buffer.size", 4 * 1024);//when
> dfsthroughput.buffer.size is configued to be 0
> private void readLocalFile(Path path, String name, Configuration conf)
> throws IOException {
> System.out.print("Reading " + name);
> resetMeasurements();
> InputStream in = new FileInputStream(new File(path.toString()));
> byte[] data = new byte[BUFFER_SIZE];
> long size = 0;
> while (size >= 0) {
> size = in.read(data);
> }
> in.close();
> printMeasurements();
> }
> {code}
> The similar case is HDFS-13513
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-13514) BenchmarkThroughput.readLocalFile hangs with misconfigured BUFFER_SIZE
[
https://issues.apache.org/jira/browse/HDFS-13514?focusedWorklogId=663238=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-663238
]
ASF GitHub Bot logged work on HDFS-13514:
-
Author: ASF GitHub Bot
Created on: 10/Oct/21 08:46
Start Date: 10/Oct/21 08:46
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #376:
URL: https://github.com/apache/hadoop/pull/376#issuecomment-939432639
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 51s | | trunk passed |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 18s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 0s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 27s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 9s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 9s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 51s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 12s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 229m 34s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 47s | | The patch does not
generate ASF License warnings. |
| | | 324m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-376/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/376 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 5a4fa0164b12 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 98fed34e44e0928c0cb75edfa681e5ca1161a993 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-376/1/testReport/ |
| Max. process+thread count | 3660 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-376/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was
