[jira] [Work logged] (HADOOP-17265) ABFS: Support for Client Correlation ID
[ https://issues.apache.org/jira/browse/HADOOP-17265?focusedWorklogId=504018=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504018 ] ASF GitHub Bot logged work on HADOOP-17265: --- Author: ASF GitHub Bot Created on: 23/Oct/20 05:04 Start Date: 23/Oct/20 05:04 Worklog Time Spent: 10m Work Description: sumangala-patki edited a comment on pull request #2344: URL: https://github.com/apache/hadoop/pull/2344#issuecomment-714398568 The yetus run error is due to tests failing on the WASB side, which is not affected by the changes proposed here. Link to [JIRA issue](https://issues.apache.org/jira/browse/HADOOP-17325). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 504018) Time Spent: 2h 50m (was: 2h 40m) > ABFS: Support for Client Correlation ID > --- > > Key: HADOOP-17265 > URL: https://issues.apache.org/jira/browse/HADOOP-17265 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sumangala Patki >Priority: Major > Labels: abfsactive, pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > Introducing a client correlation ID that appears in the Azure Storage > diagnostic logs. This will modify the client request header to include the > client-provided identifier for correlating requests. A valid correlation ID > comprises alphanumeric characters and/or hyphens, and can be up to 72 > characters in length. Invalid or missing entries for this configuration are > substituted with the empty string default. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] sumangala-patki edited a comment on pull request #2344: HADOOP-17265. ABFS: Support for Client Correlation ID
sumangala-patki edited a comment on pull request #2344: URL: https://github.com/apache/hadoop/pull/2344#issuecomment-714398568 The yetus run error is due to tests failing on the WASB side, which is not affected by the changes proposed here. Link to [JIRA issue](https://issues.apache.org/jira/browse/HADOOP-17325). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17265) ABFS: Support for Client Correlation ID
[ https://issues.apache.org/jira/browse/HADOOP-17265?focusedWorklogId=504017=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504017 ] ASF GitHub Bot logged work on HADOOP-17265: --- Author: ASF GitHub Bot Created on: 23/Oct/20 05:03 Start Date: 23/Oct/20 05:03 Worklog Time Spent: 10m Work Description: sumangala-patki edited a comment on pull request #2344: URL: https://github.com/apache/hadoop/pull/2344#issuecomment-714398568 The yetus run error is due to tests failing on the WASB side, which is not affected by the changes proposed here. Link to JIRA issue: [](https://issues.apache.org/jira/browse/HADOOP-17325) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 504017) Time Spent: 2h 40m (was: 2.5h) > ABFS: Support for Client Correlation ID > --- > > Key: HADOOP-17265 > URL: https://issues.apache.org/jira/browse/HADOOP-17265 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sumangala Patki >Priority: Major > Labels: abfsactive, pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > Introducing a client correlation ID that appears in the Azure Storage > diagnostic logs. This will modify the client request header to include the > client-provided identifier for correlating requests. A valid correlation ID > comprises alphanumeric characters and/or hyphens, and can be up to 72 > characters in length. Invalid or missing entries for this configuration are > substituted with the empty string default. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] sumangala-patki edited a comment on pull request #2344: HADOOP-17265. ABFS: Support for Client Correlation ID
sumangala-patki edited a comment on pull request #2344: URL: https://github.com/apache/hadoop/pull/2344#issuecomment-714398568 The yetus run error is due to tests failing on the WASB side, which is not affected by the changes proposed here. Link to JIRA issue: [](https://issues.apache.org/jira/browse/HADOOP-17325) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17265) ABFS: Support for Client Correlation ID
[ https://issues.apache.org/jira/browse/HADOOP-17265?focusedWorklogId=504016=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504016 ] ASF GitHub Bot logged work on HADOOP-17265: --- Author: ASF GitHub Bot Created on: 23/Oct/20 05:02 Start Date: 23/Oct/20 05:02 Worklog Time Spent: 10m Work Description: sumangala-patki edited a comment on pull request #2344: URL: https://github.com/apache/hadoop/pull/2344#issuecomment-714398568 The yetus run error is due to tests failing on the WASB side, which is not affected by the changes proposed here. Link to JIRA issue: [https://issues.apache.org/jira/browse/HADOOP-17325] This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 504016) Time Spent: 2.5h (was: 2h 20m) > ABFS: Support for Client Correlation ID > --- > > Key: HADOOP-17265 > URL: https://issues.apache.org/jira/browse/HADOOP-17265 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sumangala Patki >Priority: Major > Labels: abfsactive, pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > Introducing a client correlation ID that appears in the Azure Storage > diagnostic logs. This will modify the client request header to include the > client-provided identifier for correlating requests. A valid correlation ID > comprises alphanumeric characters and/or hyphens, and can be up to 72 > characters in length. Invalid or missing entries for this configuration are > substituted with the empty string default. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] sumangala-patki edited a comment on pull request #2344: HADOOP-17265. ABFS: Support for Client Correlation ID
sumangala-patki edited a comment on pull request #2344: URL: https://github.com/apache/hadoop/pull/2344#issuecomment-714398568 The yetus run error is due to tests failing on the WASB side, which is not affected by the changes proposed here. Link to JIRA issue: [https://issues.apache.org/jira/browse/HADOOP-17325] This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] snvijaya commented on pull request #2368: Hadoop-17296. ABFS: Force reads to be always of buffer size
snvijaya commented on pull request #2368: URL: https://github.com/apache/hadoop/pull/2368#issuecomment-714909668 Test failures reported in Yetus are not related to ABFS driver. Have created JIRA for tracking the failures. https://issues.apache.org/jira/browse/HADOOP-17325 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17325) WASB: Test failure in trunk
Sneha Vijayarajan created HADOOP-17325: -- Summary: WASB: Test failure in trunk Key: HADOOP-17325 URL: https://issues.apache.org/jira/browse/HADOOP-17325 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.0 Reporter: Sneha Vijayarajan Assignee: Esfandiar Manii WASB tests are failing in Apache trunk resulting in Yetus run failures for PRs. ||Reason||Tests|| |Failed junit tests|hadoop.fs.azure.TestNativeAzureFileSystemMocked| | |hadoop.fs.azure.TestNativeAzureFileSystemConcurrency| | |hadoop.fs.azure.TestWasbFsck| | |hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked| | |hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck| | |hadoop.fs.azure.TestNativeAzureFileSystemContractMocked| | |hadoop.fs.azure.TestOutOfBandAzureBlobOperations| | |hadoop.fs.azure.TestBlobMetadata| Many PRs are hit by this. Test report link from one of the PRs: [https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2368/5/testReport/] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2404: HDFS-15461. TestDFSClientRetries testGetFileChecksum fails.
hadoop-yetus commented on pull request #2404:
URL: https://github.com/apache/hadoop/pull/2404#issuecomment-714898064
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 13s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 45s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 1m 13s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 49s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 51s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 3m 14s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 11s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 5s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 1m 5s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 42s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 105 unchanged - 1
fixed = 105 total (was 106) |
| +1 :green_heart: | mvnsite | 1m 13s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 16m 21s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 49s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 3m 13s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 111m 45s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2404/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 203m 19s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2404/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2404 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 03e3de50482e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / da1b6e3cc28 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2404/1/testReport/ |
| Max. process+thread count | 2869 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2404/1/console |
| versions | git=2.17.1 maven=3.6.0
[GitHub] [hadoop] amahussein opened a new pull request #2405: HDFS-15461. fix testServerDefaultsWithMinimalCaching.
amahussein opened a new pull request #2405: URL: https://github.com/apache/hadoop/pull/2405 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] amahussein opened a new pull request #2404: HDFS-15461. TestDFSClientRetries testGetFileChecksum fails.
amahussein opened a new pull request #2404: URL: https://github.com/apache/hadoop/pull/2404 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] LeonGao91 commented on pull request #2288: HDFS-15548. Allow configuring DISK/ARCHIVE storage types on same device mount
LeonGao91 commented on pull request #2288: URL: https://github.com/apache/hadoop/pull/2288#issuecomment-714835959 Thanks @Jing9 For the review! 1) Very good idea, I will make a wrapper for the relationship. This will be useful if we want to update the ratio without restarting DN. 2) Another good point, I did some investigation, although "reserve" is not being reported to NN, "NonDfsUsed" will be reported and double-counted. As discussed I will add an optional field to solve this. 3) Yes, this is expected and we are looking at balancer/mover to kick-in to balance the datanode once the ratio is updated. I will put some java doc to explain this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17324) ClassNotFoundException: org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
[
https://issues.apache.org/jira/browse/HADOOP-17324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219377#comment-17219377
]
Chao Sun commented on HADOOP-17324:
---
cc [~eyang], [~rkanter] and [~ste...@apache.org]
> ClassNotFoundException:
> org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
>
>
> Key: HADOOP-17324
> URL: https://issues.apache.org/jira/browse/HADOOP-17324
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Chao Sun
>Priority: Major
>
> When downstream apps depend on {{hadoop-client-api}},
> {{hadoop-client-runtime}} and {{hadoop-client-minicluster}}, it seems the
> {{MiniYARNCluster}} could have issue because
> {{org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException}}
> is not in any of the above jars.
> {code}
> Error: Caused by: sbt.ForkMain$ForkError: java.lang.ClassNotFoundException:
> org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
> Error:at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> Error:at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
> Error:at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> Error:at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
> Error:at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:862)
> Error:at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error:at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1296)
> Error:at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:339)
> Error:at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error:at
> org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:353)
> Error:at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:127)
> Error:at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:488)
> Error:at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error:at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
> Error:at
> org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:321)
> Error:at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error:at
> org.apache.spark.deploy.yarn.BaseYarnClusterSuite.beforeAll(BaseYarnClusterSuite.scala:94)
> Error:at
> org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
> Error:at
> org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
> Error:at
> org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
> Error:at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
> Error:at
> org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
> Error:at
> org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
> Error:at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
> Error:at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> Error:at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Error:at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> Error:at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17324) ClassNotFoundException: org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
[
https://issues.apache.org/jira/browse/HADOOP-17324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated HADOOP-17324:
--
Affects Version/s: 3.3.0
> ClassNotFoundException:
> org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
>
>
> Key: HADOOP-17324
> URL: https://issues.apache.org/jira/browse/HADOOP-17324
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Chao Sun
>Priority: Major
>
> When downstream apps depend on {{hadoop-client-api}},
> {{hadoop-client-runtime}} and {{hadoop-client-minicluster}}, it seems the
> {{MiniYARNCluster}} could have issue because
> {{org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException}}
> is not in any of the above jars.
> {code}
> Error: Caused by: sbt.ForkMain$ForkError: java.lang.ClassNotFoundException:
> org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
> Error:at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> Error:at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
> Error:at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> Error:at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
> Error:at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:862)
> Error:at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error:at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1296)
> Error:at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:339)
> Error:at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error:at
> org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:353)
> Error:at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:127)
> Error:at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:488)
> Error:at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error:at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
> Error:at
> org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:321)
> Error:at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error:at
> org.apache.spark.deploy.yarn.BaseYarnClusterSuite.beforeAll(BaseYarnClusterSuite.scala:94)
> Error:at
> org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
> Error:at
> org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
> Error:at
> org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
> Error:at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
> Error:at
> org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
> Error:at
> org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
> Error:at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
> Error:at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> Error:at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Error:at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> Error:at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17314) Reduce port collisions in JUnits
[
https://issues.apache.org/jira/browse/HADOOP-17314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219362#comment-17219362
]
Ahmed Hussein commented on HADOOP-17314:
I am going to hold on this until the code becomes more stable.
> Reduce port collisions in JUnits
>
>
> Key: HADOOP-17314
> URL: https://issues.apache.org/jira/browse/HADOOP-17314
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: HADOOP-17314.001.patch
>
>
> Several Junit tests passes a non-zero port number to
> {{ServerSocketUtil.getPort()}}.
> I noticed that those non-zero values are shared between test methods. Based,
> on the implementation of {{ServerSocketUtil.getPort()}}, the port is checked
> then, the socket is closed immediately.
> Two concurrent tests can have the same port number, but one of them fails
> with binding exceptions.
> Passing a zero port number reduces the probability of returning the same port
> numbers in two different tests, because the {{ServerSocketUtil.getPort()}}
> will call a random port number.
> An example of that is as described in HDFS-15618. {{testRead()}} between
> {{TestBlockTokenWithDFSStriped}} and {{TestBlockTokenWithDFS}} where port
> 19870 is used by the two test cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17324) ClassNotFoundException: org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
Chao Sun created HADOOP-17324:
-
Summary: ClassNotFoundException:
org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
Key: HADOOP-17324
URL: https://issues.apache.org/jira/browse/HADOOP-17324
Project: Hadoop Common
Issue Type: Bug
Reporter: Chao Sun
When downstream apps depend on {{hadoop-client-api}}, {{hadoop-client-runtime}}
and {{hadoop-client-minicluster}}, it seems the {{MiniYARNCluster}} could have
issue because
{{org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException}}
is not in any of the above jars.
{code}
Error: Caused by: sbt.ForkMain$ForkError: java.lang.ClassNotFoundException:
org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
Error: at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
Error: at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
Error: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
Error: at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
Error: at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:862)
Error: at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
Error: at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1296)
Error: at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:339)
Error: at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
Error: at
org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:353)
Error: at
org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:127)
Error: at
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:488)
Error: at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
Error: at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
Error: at
org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:321)
Error: at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
Error: at
org.apache.spark.deploy.yarn.BaseYarnClusterSuite.beforeAll(BaseYarnClusterSuite.scala:94)
Error: at
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
Error: at
org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
Error: at
org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
Error: at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
Error: at
org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
Error: at
org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
Error: at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
Error: at java.util.concurrent.FutureTask.run(FutureTask.java:266)
Error: at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Error: at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Error: at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2403: HDFS-15623. Respect configured values of rpc.engine
hadoop-yetus commented on pull request #2403:
URL: https://github.com/apache/hadoop/pull/2403#issuecomment-714747564
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 12s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 34s | | trunk passed |
| +1 :green_heart: | compile | 21m 36s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 18m 13s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 49s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 50s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 19s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 15s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 52s | | the patch passed |
| +1 :green_heart: | compile | 21m 38s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 21m 38s | | the patch passed |
| +1 :green_heart: | compile | 19m 32s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 19m 32s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 42s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 18m 53s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 33s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 3m 4s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 11m 25s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 54s | | The patch does not
generate ASF License warnings. |
| | | 182m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2403/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2403 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux a592174e8dca 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7435604a91a |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2403/1/testReport/ |
| Max. process+thread count | 1373 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2403/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.1.3 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Work logged] (HADOOP-17319) Update the checkstyle config to ban some guava functions
[ https://issues.apache.org/jira/browse/HADOOP-17319?focusedWorklogId=503882=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503882 ] ASF GitHub Bot logged work on HADOOP-17319: --- Author: ASF GitHub Bot Created on: 22/Oct/20 19:28 Start Date: 22/Oct/20 19:28 Worklog Time Spent: 10m Work Description: aajisaka merged pull request #2400: URL: https://github.com/apache/hadoop/pull/2400 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 503882) Time Spent: 0.5h (was: 20m) > Update the checkstyle config to ban some guava functions > > > Key: HADOOP-17319 > URL: https://issues.apache.org/jira/browse/HADOOP-17319 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Some guava functions are banned in HADOOP-17111, HADOOP-17099, and > HADOOP-17101 via checkstyle, however, the checkstyle configuration does not > work after HADOOP-17288 because the package names have been changed. > Originally reported by [~ahussein] in HADOOP-17315. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17319) Update the checkstyle config to ban some guava functions
[ https://issues.apache.org/jira/browse/HADOOP-17319?focusedWorklogId=503883=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503883 ] ASF GitHub Bot logged work on HADOOP-17319: --- Author: ASF GitHub Bot Created on: 22/Oct/20 19:28 Start Date: 22/Oct/20 19:28 Worklog Time Spent: 10m Work Description: aajisaka commented on pull request #2400: URL: https://github.com/apache/hadoop/pull/2400#issuecomment-714712286 Thank you @ayushtkn This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 503883) Time Spent: 40m (was: 0.5h) > Update the checkstyle config to ban some guava functions > > > Key: HADOOP-17319 > URL: https://issues.apache.org/jira/browse/HADOOP-17319 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Some guava functions are banned in HADOOP-17111, HADOOP-17099, and > HADOOP-17101 via checkstyle, however, the checkstyle configuration does not > work after HADOOP-17288 because the package names have been changed. > Originally reported by [~ahussein] in HADOOP-17315. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17319) Update the checkstyle config to ban some guava functions
[ https://issues.apache.org/jira/browse/HADOOP-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-17319. Fix Version/s: 3.4.0 Resolution: Fixed Merged the PR into trunk. > Update the checkstyle config to ban some guava functions > > > Key: HADOOP-17319 > URL: https://issues.apache.org/jira/browse/HADOOP-17319 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Some guava functions are banned in HADOOP-17111, HADOOP-17099, and > HADOOP-17101 via checkstyle, however, the checkstyle configuration does not > work after HADOOP-17288 because the package names have been changed. > Originally reported by [~ahussein] in HADOOP-17315. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka merged pull request #2400: HADOOP-17319. Update the checkstyle config to ban some guava functions.
aajisaka merged pull request #2400: URL: https://github.com/apache/hadoop/pull/2400 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2400: HADOOP-17319. Update the checkstyle config to ban some guava functions.
aajisaka commented on pull request #2400: URL: https://github.com/apache/hadoop/pull/2400#issuecomment-714712286 Thank you @ayushtkn This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13203) S3A: Support fadvise "random" mode for high performance readPositioned() reads
[ https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219273#comment-17219273 ] Dongjoon Hyun commented on HADOOP-13203: Hi, [~ste...@apache.org]. `fs.s3a.experimental.input.fadvise` seems to be experimental at 3.0.0 ~ 3.3.0. Is there a reason to keep it as experimental? And, do we have a roadmap to make it `fs.s3a.input.fadvise` in the future releases? > S3A: Support fadvise "random" mode for high performance readPositioned() reads > -- > > Key: HADOOP-13203 > URL: https://issues.apache.org/jira/browse/HADOOP-13203 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.0 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Major > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: HADOOP-13203-branch-2-001.patch, > HADOOP-13203-branch-2-002.patch, HADOOP-13203-branch-2-003.patch, > HADOOP-13203-branch-2-004.patch, HADOOP-13203-branch-2-005.patch, > HADOOP-13203-branch-2-006.patch, HADOOP-13203-branch-2-007.patch, > HADOOP-13203-branch-2-008.patch, HADOOP-13203-branch-2-009.patch, > HADOOP-13203-branch-2-010.patch, stream_stats.tar.gz > > > Currently file's "contentLength" is set as the "requestedStreamLen", when > invoking S3AInputStream::reopen(). As a part of lazySeek(), sometimes the > stream had to be closed and reopened. But lots of times the stream was closed > with abort() causing the internal http connection to be unusable. This incurs > lots of connection establishment cost in some jobs. It would be good to set > the correct value for the stream length to avoid connection aborts. > I will post the patch once aws tests passes in my machine. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17323) s3a getFileStatus("/") to skip IO
Steve Loughran created HADOOP-17323:
---
Summary: s3a getFileStatus("/") to skip IO
Key: HADOOP-17323
URL: https://issues.apache.org/jira/browse/HADOOP-17323
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: 3.3.0
Reporter: Steve Loughran
Calls to "hadoop fs ls s3a://something/" add a full getFileStatus() sequence
(HEAD, LIST, LIST) because FsShell Ls calls getFileStatus on every input.
We should just build a root status entry immediately. There's one consequence:
if a user has disabled the s3a existence probe, there will be no checks for the
bucket existence. But that can be viewed as a "well, that's what happens"
behaviour
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?focusedWorklogId=503843=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503843
]
ASF GitHub Bot logged work on HADOOP-17175:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 18:16
Start Date: 22/Oct/20 18:16
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714671354
Merged. Thank you @tasanuma
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503843)
Time Spent: 2h 20m (was: 2h 10m)
> [JDK11] Fix javadoc errors in hadoop-common module
> --
>
> Key: HADOOP-17175
> URL: https://issues.apache.org/jira/browse/HADOOP-17175
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Full error log:
> https://gist.github.com/aajisaka/5504dd485cc4bc98a6a245830d450919
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java:57:
> error: unexpected end tag:
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedTimeCostProvider.java:33:
> error: self-closing element not allowed
> [ERROR] * This allows for configuration of how heavily each of the
> operations
> [ERROR]^
> ...
> {noformat}
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2397: HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module.
aajisaka commented on pull request #2397: URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714671354 Merged. Thank you @tasanuma This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-17175:
---
Fix Version/s: 3.4.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> [JDK11] Fix javadoc errors in hadoop-common module
> --
>
> Key: HADOOP-17175
> URL: https://issues.apache.org/jira/browse/HADOOP-17175
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Full error log:
> https://gist.github.com/aajisaka/5504dd485cc4bc98a6a245830d450919
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java:57:
> error: unexpected end tag:
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedTimeCostProvider.java:33:
> error: self-closing element not allowed
> [ERROR] * This allows for configuration of how heavily each of the
> operations
> [ERROR]^
> ...
> {noformat}
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?focusedWorklogId=503842=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503842
]
ASF GitHub Bot logged work on HADOOP-17175:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 18:15
Start Date: 22/Oct/20 18:15
Worklog Time Spent: 10m
Work Description: aajisaka merged pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503842)
Time Spent: 2h 10m (was: 2h)
> [JDK11] Fix javadoc errors in hadoop-common module
> --
>
> Key: HADOOP-17175
> URL: https://issues.apache.org/jira/browse/HADOOP-17175
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Full error log:
> https://gist.github.com/aajisaka/5504dd485cc4bc98a6a245830d450919
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java:57:
> error: unexpected end tag:
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedTimeCostProvider.java:33:
> error: self-closing element not allowed
> [ERROR] * This allows for configuration of how heavily each of the
> operations
> [ERROR]^
> ...
> {noformat}
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka merged pull request #2397: HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module.
aajisaka merged pull request #2397: URL: https://github.com/apache/hadoop/pull/2397 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2397: HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module.
aajisaka commented on pull request #2397: URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714666546 TestZKFailover#testNoZK failed but it is not related to the patch. Filed https://issues.apache.org/jira/browse/HADOOP-17322 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?focusedWorklogId=503839=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503839
]
ASF GitHub Bot logged work on HADOOP-17175:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 18:07
Start Date: 22/Oct/20 18:07
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714666546
TestZKFailover#testNoZK failed but it is not related to the patch. Filed
https://issues.apache.org/jira/browse/HADOOP-17322
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503839)
Time Spent: 2h (was: 1h 50m)
> [JDK11] Fix javadoc errors in hadoop-common module
> --
>
> Key: HADOOP-17175
> URL: https://issues.apache.org/jira/browse/HADOOP-17175
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Full error log:
> https://gist.github.com/aajisaka/5504dd485cc4bc98a6a245830d450919
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java:57:
> error: unexpected end tag:
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedTimeCostProvider.java:33:
> error: self-closing element not allowed
> [ERROR] * This allows for configuration of how heavily each of the
> operations
> [ERROR]^
> ...
> {noformat}
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17322) TestZKFailoverController#testNoZK is flaky
[
https://issues.apache.org/jira/browse/HADOOP-17322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219240#comment-17219240
]
Akira Ajisaka commented on HADOOP-17322:
The test assumes there are no ZooKeeper services and fail in the following code:
{code:title=ZKFailoverController#doRun(String[])}
try {
initZK();
} catch (KeeperException ke) {
LOG.error("Unable to start failover controller. Unable to connect "
+ "to ZooKeeper quorum at " + zkQuorum + ". Please check the "
+ "configured value for " + ZK_QUORUM_KEY + " and ensure that "
+ "ZooKeeper is running.", ke);
return ERR_CODE_NO_ZK;
}
{code}
however, the error message shows that {{initZK()}} passed and failed later. A
ZK service existed for some reason.
I'd suggest we should disable parallel tests in hadoop-common.
> TestZKFailoverController#testNoZK is flaky
> --
>
> Key: HADOOP-17322
> URL: https://issues.apache.org/jira/browse/HADOOP-17322
> Project: Hadoop Common
> Issue Type: Bug
> Components: ha, test
>Reporter: Akira Ajisaka
>Priority: Major
>
> TestZKFailoverController#testNoZK failed in
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/4/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
> {noformat}
> [ERROR] testNoZK(org.apache.hadoop.ha.TestZKFailoverController) Time
> elapsed: 0.114 s <<< ERROR!
> java.lang.IllegalStateException
> at
> org.apache.hadoop.thirdparty.com.google.common.base.Preconditions.checkState(Preconditions.java:492)
> at
> org.apache.hadoop.ha.ActiveStandbyElector.parentZNodeExists(ActiveStandbyElector.java:318)
> at
> org.apache.hadoop.ha.ZKFailoverController.doRun(ZKFailoverController.java:230)
> at
> org.apache.hadoop.ha.ZKFailoverController.access$000(ZKFailoverController.java:60)
> at
> org.apache.hadoop.ha.ZKFailoverController$1.run(ZKFailoverController.java:178)
> at
> org.apache.hadoop.ha.ZKFailoverController$1.run(ZKFailoverController.java:174)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:485)
> at
> org.apache.hadoop.ha.ZKFailoverController.run(ZKFailoverController.java:174)
> at
> org.apache.hadoop.ha.TestZKFailoverController.runFC(TestZKFailoverController.java:700)
> at
> org.apache.hadoop.ha.TestZKFailoverController.testNoZK(TestZKFailoverController.java:135)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:80)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17322) TestZKFailoverController#testNoZK is flaky
Akira Ajisaka created HADOOP-17322:
--
Summary: TestZKFailoverController#testNoZK is flaky
Key: HADOOP-17322
URL: https://issues.apache.org/jira/browse/HADOOP-17322
Project: Hadoop Common
Issue Type: Bug
Components: ha, test
Reporter: Akira Ajisaka
TestZKFailoverController#testNoZK failed in
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/4/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
{noformat}
[ERROR] testNoZK(org.apache.hadoop.ha.TestZKFailoverController) Time elapsed:
0.114 s <<< ERROR!
java.lang.IllegalStateException
at
org.apache.hadoop.thirdparty.com.google.common.base.Preconditions.checkState(Preconditions.java:492)
at
org.apache.hadoop.ha.ActiveStandbyElector.parentZNodeExists(ActiveStandbyElector.java:318)
at
org.apache.hadoop.ha.ZKFailoverController.doRun(ZKFailoverController.java:230)
at
org.apache.hadoop.ha.ZKFailoverController.access$000(ZKFailoverController.java:60)
at
org.apache.hadoop.ha.ZKFailoverController$1.run(ZKFailoverController.java:178)
at
org.apache.hadoop.ha.ZKFailoverController$1.run(ZKFailoverController.java:174)
at
org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:485)
at
org.apache.hadoop.ha.ZKFailoverController.run(ZKFailoverController.java:174)
at
org.apache.hadoop.ha.TestZKFailoverController.runFC(TestZKFailoverController.java:700)
at
org.apache.hadoop.ha.TestZKFailoverController.testNoZK(TestZKFailoverController.java:135)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:80)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work started] (HADOOP-17320) Update apache/hadoop:3 to 3.3.0 release
[
https://issues.apache.org/jira/browse/HADOOP-17320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HADOOP-17320 started by Attila Doroszlai.
-
> Update apache/hadoop:3 to 3.3.0 release
> ---
>
> Key: HADOOP-17320
> URL: https://issues.apache.org/jira/browse/HADOOP-17320
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
>
> {{apache/hadoop:3}} docker image should be updated to the [Hadoop 3.3.0
> release|https://hadoop.apache.org/release/3.3.0.html].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2377: HDFS-15624. fix the function of setting quota by storage type
hadoop-yetus commented on pull request #2377:
URL: https://github.com/apache/hadoop/pull/2377#issuecomment-714651305
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 5s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 11m 18s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 22m 56s | | trunk passed |
| +1 :green_heart: | compile | 21m 28s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 18m 7s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 33s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 13s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 59s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 19s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 6m 49s | | trunk passed |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 36s | | the patch passed |
| +1 :green_heart: | compile | 20m 53s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 20m 53s | | the patch passed |
| +1 :green_heart: | compile | 18m 9s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 18m 9s | | the patch passed |
| +1 :green_heart: | checkstyle | 2m 52s | | root: The patch generated
0 new + 734 unchanged - 1 fixed = 734 total (was 735) |
| +1 :green_heart: | mvnsite | 3m 34s | | the patch passed |
| -1 :x: | whitespace | 0m 0s |
[/whitespace-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/10/artifact/out/whitespace-eol.txt)
| The patch has 2 line(s) that end in whitespace. Use git apply
--whitespace=fix <>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | shadedclient | 16m 37s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 13s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 4s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 7m 17s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 9m 33s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 115m 15s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/10/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | unit | 10m 37s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 55s | | The patch does not
generate ASF License warnings. |
| | | 331m 15s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes |
| |
hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
| | hadoop.hdfs.TestFileChecksum |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2377 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 60ae6905fd3e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7435604a91a
[GitHub] [hadoop] hchaverri opened a new pull request #2403: HDFS-15623. Respect configured values of rpc.engine
hchaverri opened a new pull request #2403: URL: https://github.com/apache/hadoop/pull/2403 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage
[
https://issues.apache.org/jira/browse/HADOOP-17201?focusedWorklogId=503811=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503811
]
ASF GitHub Bot logged work on HADOOP-17201:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 17:13
Start Date: 22/Oct/20 17:13
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2402:
URL: https://github.com/apache/hadoop/pull/2402#issuecomment-714636636
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 43s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 37s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 13s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 9s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 10s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 12s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 15s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 77m 24s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2402/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2402 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux b615f35ec1fe 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7435604a91a |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2402/1/testReport/ |
| Max. process+thread count | 458 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2402/1/console |
[GitHub] [hadoop] hadoop-yetus commented on pull request #2402: HADOOP-17201. eliminate double retry around delete/finishedWrite
hadoop-yetus commented on pull request #2402:
URL: https://github.com/apache/hadoop/pull/2402#issuecomment-714636636
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 43s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 37s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 13s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 9s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 10s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 12s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 15s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 77m 24s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2402/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2402 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux b615f35ec1fe 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7435604a91a |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2402/1/testReport/ |
| Max. process+thread count | 458 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2402/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.1.3 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For
[GitHub] [hadoop] hadoop-yetus commented on pull request #2368: Hadoop-17296. ABFS: Force reads to be always of buffer size
hadoop-yetus commented on pull request #2368:
URL: https://github.com/apache/hadoop/pull/2368#issuecomment-714616055
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 8s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 46s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 59s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 56s | | trunk passed |
| -0 :warning: | patch | 1m 18s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 25s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 0m 57s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 1m 28s |
[/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2368/5/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 73m 57s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.azure.TestNativeAzureFileSystemMocked |
| | hadoop.fs.azure.TestNativeAzureFileSystemConcurrency |
| | hadoop.fs.azure.TestWasbFsck |
| | hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked |
| | hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck |
| | hadoop.fs.azure.TestNativeAzureFileSystemContractMocked |
| | hadoop.fs.azure.TestOutOfBandAzureBlobOperations |
| | hadoop.fs.azure.TestBlobMetadata |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2368/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2368 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle markdownlint |
| uname | Linux ec475f03c972 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7435604a91a |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
[GitHub] [hadoop] liuml07 commented on pull request #2377: HDFS-15624. fix the function of setting quota by storage type
liuml07 commented on pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#issuecomment-714616321 > Better we have the NamenodeLayout also fixed and conclude the Nvdimm stuff. Yes, I think I'm +1 on this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage
[
https://issues.apache.org/jira/browse/HADOOP-17201?focusedWorklogId=503780=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503780
]
ASF GitHub Bot logged work on HADOOP-17201:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 15:53
Start Date: 22/Oct/20 15:53
Worklog Time Spent: 10m
Work Description: steveloughran opened a new pull request #2402:
URL: https://github.com/apache/hadoop/pull/2402
make putObject & putObjectDirect retrying everywhere, update @RetryPolicy
annotation, and make sure callers are not attempting to retry the methods
test failure related to inconistent s3 client creating too many throttle
events for the retrier to handle...need to look @ my settings to see if I've
turned off the AWS SDK retries there
```
[ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
60.785 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.commit.ITestCommitOperations
[ERROR]
testCommitEmptyFile(org.apache.hadoop.fs.s3a.commit.ITestCommitOperations)
Time elapsed: 3.289 s <<< ERROR!
com.amazonaws.AmazonServiceException: throttled count = 1 (Service: null;
Status Code: 503; Error Code: null; Request ID: null)
at
org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client.maybeFail(InconsistentAmazonS3Client.java:571)
at
org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client.maybeFail(InconsistentAmazonS3Client.java:586)
at
org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client.putObject(InconsistentAmazonS3Client.java:226)
at
com.amazonaws.services.s3.transfer.internal.UploadCallable.uploadInOneChunk(UploadCallable.java:131)
at
com.amazonaws.services.s3.transfer.internal.UploadCallable.call(UploadCallable.java:123)
at
com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:143)
at
com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:48)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503780)
Remaining Estimate: 0h
Time Spent: 10m
> Spark job with s3acommitter stuck at the last stage
> ---
>
> Key: HADOOP-17201
> URL: https://issues.apache.org/jira/browse/HADOOP-17201
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.2.1
> Environment: we are on spark 2.4.5/hadoop 3.2.1 with s3a committer.
> spark.hadoop.fs.s3a.committer.magic.enabled: 'true'
> spark.hadoop.fs.s3a.committer.name: magic
>Reporter: Dyno
>Priority: Major
> Attachments: exec-120.log, exec-125.log, exec-25.log, exec-31.log,
> exec-36.log, exec-44.log, exec-5.log, exec-64.log, exec-7.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> usually our spark job took 1 hour or 2 to finish, occasionally it runs for
> more than 3 hour and then we know it's stuck and usually the executor has
> stack like this
> {{
> "Executor task launch worker for task 78620" #265 daemon prio=5 os_prio=0
> tid=0x7f73e0005000 nid=0x12d waiting on condition [0x7f74cb291000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:349)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:285)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteObjects(S3AFileSystem.java:1457)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.removeKeys(S3AFileSystem.java:1717)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteUnnecessaryFakeDirectories(S3AFileSystem.java:2785)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.finishedWrite(S3AFileSystem.java:2751)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$finalizeMultipartUpload$1(WriteOperationHelper.java:238)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper$$Lambda$210/1059071691.execute(Unknown
> Source)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
> at
>
[jira] [Updated] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage
[
https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HADOOP-17201:
Labels: pull-request-available (was: )
> Spark job with s3acommitter stuck at the last stage
> ---
>
> Key: HADOOP-17201
> URL: https://issues.apache.org/jira/browse/HADOOP-17201
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.2.1
> Environment: we are on spark 2.4.5/hadoop 3.2.1 with s3a committer.
> spark.hadoop.fs.s3a.committer.magic.enabled: 'true'
> spark.hadoop.fs.s3a.committer.name: magic
>Reporter: Dyno
>Priority: Major
> Labels: pull-request-available
> Attachments: exec-120.log, exec-125.log, exec-25.log, exec-31.log,
> exec-36.log, exec-44.log, exec-5.log, exec-64.log, exec-7.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> usually our spark job took 1 hour or 2 to finish, occasionally it runs for
> more than 3 hour and then we know it's stuck and usually the executor has
> stack like this
> {{
> "Executor task launch worker for task 78620" #265 daemon prio=5 os_prio=0
> tid=0x7f73e0005000 nid=0x12d waiting on condition [0x7f74cb291000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:349)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:285)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteObjects(S3AFileSystem.java:1457)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.removeKeys(S3AFileSystem.java:1717)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteUnnecessaryFakeDirectories(S3AFileSystem.java:2785)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.finishedWrite(S3AFileSystem.java:2751)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$finalizeMultipartUpload$1(WriteOperationHelper.java:238)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper$$Lambda$210/1059071691.execute(Unknown
> Source)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
> at
> org.apache.hadoop.fs.s3a.Invoker$$Lambda$23/586859139.execute(Unknown Source)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.finalizeMultipartUpload(WriteOperationHelper.java:226)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.completeMPUwithRetries(WriteOperationHelper.java:271)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.complete(S3ABlockOutputStream.java:660)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.access$200(S3ABlockOutputStream.java:521)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream.close(S3ABlockOutputStream.java:385)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
> at
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101)
> at
> org.apache.parquet.hadoop.util.HadoopPositionOutputStream.close(HadoopPositionOutputStream.java:64)
> at
> org.apache.parquet.hadoop.ParquetFileWriter.end(ParquetFileWriter.java:685)
> at
> org.apache.parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:122)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:165)
> at
> org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:42)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:57)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:74)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask$3.apply(FileFormatWriter.scala:247)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask$3.apply(FileFormatWriter.scala:242)
> at
> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1394)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$.org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask(FileFormatWriter.scala:248)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$write$1.apply(FileFormatWriter.scala:170)
> at
>
[GitHub] [hadoop] steveloughran opened a new pull request #2402: HADOOP-17201. eliminate double retry around delete/finishedWrite
steveloughran opened a new pull request #2402: URL: https://github.com/apache/hadoop/pull/2402 make putObject & putObjectDirect retrying everywhere, update @RetryPolicy annotation, and make sure callers are not attempting to retry the methods test failure related to inconistent s3 client creating too many throttle events for the retrier to handle...need to look @ my settings to see if I've turned off the AWS SDK retries there ``` [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 60.785 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.commit.ITestCommitOperations [ERROR] testCommitEmptyFile(org.apache.hadoop.fs.s3a.commit.ITestCommitOperations) Time elapsed: 3.289 s <<< ERROR! com.amazonaws.AmazonServiceException: throttled count = 1 (Service: null; Status Code: 503; Error Code: null; Request ID: null) at org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client.maybeFail(InconsistentAmazonS3Client.java:571) at org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client.maybeFail(InconsistentAmazonS3Client.java:586) at org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client.putObject(InconsistentAmazonS3Client.java:226) at com.amazonaws.services.s3.transfer.internal.UploadCallable.uploadInOneChunk(UploadCallable.java:131) at com.amazonaws.services.s3.transfer.internal.UploadCallable.call(UploadCallable.java:123) at com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:143) at com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:48) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16780) Track unstable tests according to aarch CI due to OOM
[ https://issues.apache.org/jira/browse/HADOOP-16780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein resolved HADOOP-16780. Resolution: Resolved > Track unstable tests according to aarch CI due to OOM > - > > Key: HADOOP-16780 > URL: https://issues.apache.org/jira/browse/HADOOP-16780 > Project: Hadoop Common > Issue Type: Bug >Reporter: liusheng >Priority: Major > > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestDFSClientRetries.testLeaseRenewSocketTimeout|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestDFSClientRetries/testLeaseRenewSocketTimeout/]|1.9 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksum1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksum1/]|2 > min 53 sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksum3|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksum3/]|2 > min 13 sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery4|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery4/]|1 > min 32 sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery5|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery5/]|11 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery6|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery6/]|3.8 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery7|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery7/]|7.7 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery8|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery8/]|3.7 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery9|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery9/]|4 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksumCompositeCrc.testStripedFileChecksumWithMissedDataBlocksRangeQuery6|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksumCompositeCrc/testStripedFileChecksumWithMissedDataBlocksRangeQuery6/]|10 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! >
[jira] [Reopened] (HADOOP-16780) Track unstable tests according to aarch CI due to OOM
[ https://issues.apache.org/jira/browse/HADOOP-16780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein reopened HADOOP-16780: > Track unstable tests according to aarch CI due to OOM > - > > Key: HADOOP-16780 > URL: https://issues.apache.org/jira/browse/HADOOP-16780 > Project: Hadoop Common > Issue Type: Bug >Reporter: liusheng >Priority: Major > > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestDFSClientRetries.testLeaseRenewSocketTimeout|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestDFSClientRetries/testLeaseRenewSocketTimeout/]|1.9 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksum1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksum1/]|2 > min 53 sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksum3|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksum3/]|2 > min 13 sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery4|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery4/]|1 > min 32 sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery5|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery5/]|11 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery6|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery6/]|3.8 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery7|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery7/]|7.7 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery8|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery8/]|3.7 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery9|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksum/testStripedFileChecksumWithMissedDataBlocksRangeQuery9/]|4 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! > [org.apache.hadoop.hdfs.TestFileChecksumCompositeCrc.testStripedFileChecksumWithMissedDataBlocksRangeQuery6|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/testReport/org.apache.hadoop.hdfs/TestFileChecksumCompositeCrc/testStripedFileChecksumWithMissedDataBlocksRangeQuery6/]|10 > sec|[1|https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/55/]| > |!https://builds.apache.org/static/8ad18952/images/16x16/document_add.png! >
[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage
[
https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219058#comment-17219058
]
Steve Loughran commented on HADOOP-17201:
-
...OK. the HADOOP-17244 patch will eliminate the double retry. It doesn't do
anything about identifying/fixing the problem you are seeing or downgrading.
FWIW, that downgrade does run the risk of hiding regressions in the s3a code,
e.g HADOOP-17261. At least now we can see when we've got something wrong. Nice
idea tho'
[~james...@ymail.com] if you could try turning off s3a retries entirely
fs.s3a.retry.limit = 1) that would help get us your stack trace. This doesn't
affect throttling...retries there are handled in the AWS SDK itself. Depending
on the hadoop build you have, it does affect how many attempts we do waiting
for newly created s3 objects to be visible. If you see 404s on rename then go
to 2 and a longer initial retry interval (long story)
> Spark job with s3acommitter stuck at the last stage
> ---
>
> Key: HADOOP-17201
> URL: https://issues.apache.org/jira/browse/HADOOP-17201
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.2.1
> Environment: we are on spark 2.4.5/hadoop 3.2.1 with s3a committer.
> spark.hadoop.fs.s3a.committer.magic.enabled: 'true'
> spark.hadoop.fs.s3a.committer.name: magic
>Reporter: Dyno
>Priority: Major
> Attachments: exec-120.log, exec-125.log, exec-25.log, exec-31.log,
> exec-36.log, exec-44.log, exec-5.log, exec-64.log, exec-7.log
>
>
> usually our spark job took 1 hour or 2 to finish, occasionally it runs for
> more than 3 hour and then we know it's stuck and usually the executor has
> stack like this
> {{
> "Executor task launch worker for task 78620" #265 daemon prio=5 os_prio=0
> tid=0x7f73e0005000 nid=0x12d waiting on condition [0x7f74cb291000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:349)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:285)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteObjects(S3AFileSystem.java:1457)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.removeKeys(S3AFileSystem.java:1717)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteUnnecessaryFakeDirectories(S3AFileSystem.java:2785)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.finishedWrite(S3AFileSystem.java:2751)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$finalizeMultipartUpload$1(WriteOperationHelper.java:238)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper$$Lambda$210/1059071691.execute(Unknown
> Source)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
> at
> org.apache.hadoop.fs.s3a.Invoker$$Lambda$23/586859139.execute(Unknown Source)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.finalizeMultipartUpload(WriteOperationHelper.java:226)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.completeMPUwithRetries(WriteOperationHelper.java:271)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.complete(S3ABlockOutputStream.java:660)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.access$200(S3ABlockOutputStream.java:521)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream.close(S3ABlockOutputStream.java:385)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
> at
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101)
> at
> org.apache.parquet.hadoop.util.HadoopPositionOutputStream.close(HadoopPositionOutputStream.java:64)
> at
> org.apache.parquet.hadoop.ParquetFileWriter.end(ParquetFileWriter.java:685)
> at
> org.apache.parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:122)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:165)
> at
> org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:42)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:57)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:74)
> at
>
[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage
[
https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219036#comment-17219036
]
Steve Loughran commented on HADOOP-17201:
-
update. Yes, we're doing a retry in WriteOperationsHelper in putObjectDirect,
but that's because the put() isn't retrying. Delete is and its that which isn't
working.
If it makes you feel better, Hadoop 3.3.1 is going to let you keep directory
markers, so no delete at that point, at all
> Spark job with s3acommitter stuck at the last stage
> ---
>
> Key: HADOOP-17201
> URL: https://issues.apache.org/jira/browse/HADOOP-17201
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.2.1
> Environment: we are on spark 2.4.5/hadoop 3.2.1 with s3a committer.
> spark.hadoop.fs.s3a.committer.magic.enabled: 'true'
> spark.hadoop.fs.s3a.committer.name: magic
>Reporter: Dyno
>Priority: Major
> Attachments: exec-120.log, exec-125.log, exec-25.log, exec-31.log,
> exec-36.log, exec-44.log, exec-5.log, exec-64.log, exec-7.log
>
>
> usually our spark job took 1 hour or 2 to finish, occasionally it runs for
> more than 3 hour and then we know it's stuck and usually the executor has
> stack like this
> {{
> "Executor task launch worker for task 78620" #265 daemon prio=5 os_prio=0
> tid=0x7f73e0005000 nid=0x12d waiting on condition [0x7f74cb291000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:349)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:285)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteObjects(S3AFileSystem.java:1457)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.removeKeys(S3AFileSystem.java:1717)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteUnnecessaryFakeDirectories(S3AFileSystem.java:2785)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.finishedWrite(S3AFileSystem.java:2751)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$finalizeMultipartUpload$1(WriteOperationHelper.java:238)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper$$Lambda$210/1059071691.execute(Unknown
> Source)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
> at
> org.apache.hadoop.fs.s3a.Invoker$$Lambda$23/586859139.execute(Unknown Source)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.finalizeMultipartUpload(WriteOperationHelper.java:226)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.completeMPUwithRetries(WriteOperationHelper.java:271)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.complete(S3ABlockOutputStream.java:660)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.access$200(S3ABlockOutputStream.java:521)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream.close(S3ABlockOutputStream.java:385)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
> at
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101)
> at
> org.apache.parquet.hadoop.util.HadoopPositionOutputStream.close(HadoopPositionOutputStream.java:64)
> at
> org.apache.parquet.hadoop.ParquetFileWriter.end(ParquetFileWriter.java:685)
> at
> org.apache.parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:122)
> at
> org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:165)
> at
> org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:42)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:57)
> at
> org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:74)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask$3.apply(FileFormatWriter.scala:247)
> at
> org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask$3.apply(FileFormatWriter.scala:242)
> at
> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1394)
> at
>
[GitHub] [hadoop] hadoop-yetus commented on pull request #2309: HDFS-15580. [JDK 12] DFSTestUtil#addDataNodeLayoutVersion fails
hadoop-yetus commented on pull request #2309:
URL: https://github.com/apache/hadoop/pull/2309#issuecomment-714524923
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 9s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 5s | | trunk passed |
| +1 :green_heart: | compile | 1m 18s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 1m 12s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 11s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 3m 4s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 3s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 1m 9s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 1m 9s | | the patch passed |
| +1 :green_heart: | compile | 1m 5s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 1m 5s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 51s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 458 unchanged - 13
fixed = 458 total (was 471) |
| +1 :green_heart: | mvnsite | 1m 11s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 48s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 22s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 3m 9s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 158m 7s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2309/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not
generate ASF License warnings. |
| | | 245m 8s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| | hadoop.hdfs.TestDecommissionWithStriped |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.TestModTime |
| | hadoop.hdfs.TestReconstructStripedFile |
| | hadoop.hdfs.server.balancer.TestBalancerRPCDelay |
| | hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate |
| | hadoop.hdfs.TestDistributedFileSystemWithECFileWithRandomECPolicy |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.tools.TestDFSAdminWithHA |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
| | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2309/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2309 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux cd48e3df97c5 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh
[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage
[
https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219029#comment-17219029
]
Steve Loughran commented on HADOOP-17201:
-
bq. Regarding the bulk delete call failure, I do have a suggestion (learned
from our AWS representative): when the bulk delete fails and throws the
MultiObjectDeleteException, try to collect all the failed ones and delete them
individually as a second chance. This pattern is really useful and has been
working for us. I think S3aFileSystem should implement the same logic as a
configurable option, in additional to the current behavior which is if one or
some objects failed the delete, fail the whole multi objects delete. That
should make the deleteObjects() call much more resilient to the S3 intermittent
delete issue.
Oh, that is a good idea. Nice.
Not going to help with things blocking in retry. What code do we have which
things the failing error is going to get better?
bq. we don't know whether the last stuck task will eventually finish
successfully or fail because we don't have the patient and money to let it.
I think something is doing retry around retry.
you could cut back on the retry count/intervals ("fs.s3a.retry.limit") default:
7. Cut it down to 2 or 3 for some retries but you don't get the 7 * 7 retry
sequence, just 2 * 2 -but still with some recovery from transient failures.
I like the idea of fallback here.
I've currently got HADOOP-17244 up in my IDE...which does go near that handling
of multidelete errors. I won't do the fallback, but I will review to make sure
there's no double delete
> Spark job with s3acommitter stuck at the last stage
> ---
>
> Key: HADOOP-17201
> URL: https://issues.apache.org/jira/browse/HADOOP-17201
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.2.1
> Environment: we are on spark 2.4.5/hadoop 3.2.1 with s3a committer.
> spark.hadoop.fs.s3a.committer.magic.enabled: 'true'
> spark.hadoop.fs.s3a.committer.name: magic
>Reporter: Dyno
>Priority: Major
> Attachments: exec-120.log, exec-125.log, exec-25.log, exec-31.log,
> exec-36.log, exec-44.log, exec-5.log, exec-64.log, exec-7.log
>
>
> usually our spark job took 1 hour or 2 to finish, occasionally it runs for
> more than 3 hour and then we know it's stuck and usually the executor has
> stack like this
> {{
> "Executor task launch worker for task 78620" #265 daemon prio=5 os_prio=0
> tid=0x7f73e0005000 nid=0x12d waiting on condition [0x7f74cb291000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:349)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:285)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteObjects(S3AFileSystem.java:1457)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.removeKeys(S3AFileSystem.java:1717)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.deleteUnnecessaryFakeDirectories(S3AFileSystem.java:2785)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.finishedWrite(S3AFileSystem.java:2751)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$finalizeMultipartUpload$1(WriteOperationHelper.java:238)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper$$Lambda$210/1059071691.execute(Unknown
> Source)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
> at
> org.apache.hadoop.fs.s3a.Invoker$$Lambda$23/586859139.execute(Unknown Source)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.finalizeMultipartUpload(WriteOperationHelper.java:226)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.completeMPUwithRetries(WriteOperationHelper.java:271)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.complete(S3ABlockOutputStream.java:660)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.access$200(S3ABlockOutputStream.java:521)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream.close(S3ABlockOutputStream.java:385)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
> at
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101)
> at
> org.apache.parquet.hadoop.util.HadoopPositionOutputStream.close(HadoopPositionOutputStream.java:64)
> at
> org.apache.parquet.hadoop.ParquetFileWriter.end(ParquetFileWriter.java:685)
> at
>
[GitHub] [hadoop] hadoop-yetus commented on pull request #2390: YARN-10442. RM should make sure node label file highly available.
hadoop-yetus commented on pull request #2390: URL: https://github.com/apache/hadoop/pull/2390#issuecomment-714503641 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 0s | | Docker mode activated. | | -1 :x: | docker | 2m 45s | | Docker failed to build yetus/hadoop:3fc007a37b1. | | Subsystem | Report/Notes | |--:|:-| | JIRA Issue | YARN-10442 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/13013592/YARN-10442.002.patch | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2390/5/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2390: YARN-10442. RM should make sure node label file highly available.
hadoop-yetus commented on pull request #2390:
URL: https://github.com/apache/hadoop/pull/2390#issuecomment-714480787
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 1s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 11m 33s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 10s | | trunk passed |
| +1 :green_heart: | compile | 8m 53s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 7m 36s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 1m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 51s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 32s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 35s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 57s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 5m 53s | | trunk passed |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 9s | | the patch passed |
| +1 :green_heart: | compile | 8m 13s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 8m 13s | | the patch passed |
| +1 :green_heart: | compile | 7m 37s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 7m 37s | | the patch passed |
| -0 :warning: | checkstyle | 1m 26s |
[/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2390/4/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 3 new + 210 unchanged
- 0 fixed = 213 total (was 210) |
| +1 :green_heart: | mvnsite | 2m 51s | | the patch passed |
| -1 :x: | whitespace | 0m 0s |
[/whitespace-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2390/4/artifact/out/whitespace-eol.txt)
| The patch has 2 line(s) that end in whitespace. Use git apply
--whitespace=fix <>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | shadedclient | 15m 3s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 18s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 2m 21s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 6m 12s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 1m 4s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2390/4/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt)
| hadoop-yarn-api in the patch passed. |
| +1 :green_heart: | unit | 4m 12s | | hadoop-yarn-common in the patch
passed. |
| -1 :x: | unit | 89m 50s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2390/4/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
| hadoop-yarn-server-resourcemanager in the patch passed. |
| -1 :x: | asflicense | 0m 52s |
[/patch-asflicense-problems.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2390/4/artifact/out/patch-asflicense-problems.txt)
| The patch generated 1 ASF License warnings. |
| | | 228m 43s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
| |
hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
[jira] [Work logged] (HADOOP-17244) HADOOP-17244. S3A directory delete tombstones dir markers prematurely.
[
https://issues.apache.org/jira/browse/HADOOP-17244?focusedWorklogId=503715=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503715
]
ASF GitHub Bot logged work on HADOOP-17244:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 12:56
Start Date: 22/Oct/20 12:56
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #2310:
URL: https://github.com/apache/hadoop/pull/2310#issuecomment-714475083
Checkstyle still fussy. What a pain
```
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:93:
"Bulk delete operation failed to delete all objects; failure count =
{}",: Line is longer than 80 characters (found 81). [LineLength]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:331:
List undeleted = extractUndeletedKeyPaths(deleteException,
qualifier);: Line is longer than 80 characters (found 83). [LineLength]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:360:
public final String key;:25: Variable 'key' must be private and have
accessor methods. [VisibilityModifier]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:361:
public final Path path;:23: Variable 'path' must be private and have
accessor methods. [VisibilityModifier]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:362:
public final boolean directoryMarker;:26: Variable 'directoryMarker' must
be private and have accessor methods. [VisibilityModifier]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:398:
if (this == o) { return true; }:22: '{' at column 22 should have line
break after. [LeftCurly]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:399:
if (o == null || getClass() != o.getClass()) { return false; }:52: '{' at
column 52 should have line break after. [LeftCurly]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestPartialRenamesDeletes.java:668:
MetricDiff deleteObjectCount = new MetricDiff(roleFS,
OBJECT_DELETE_OBJECTS);: Line is longer than 80 characters (found 81).
[LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestPartialRenamesDeletes.java:676:
deleteObjectCount.assertDiffEquals("Wrong count of objects in delete
request",: Line is longer than 80 characters (found 84). [LineLength]
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503715)
Time Spent: 4h (was: 3h 50m)
> HADOOP-17244. S3A directory delete tombstones dir markers prematurely.
> --
>
> Key: HADOOP-17244
> URL: https://issues.apache.org/jira/browse/HADOOP-17244
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.1
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 3.3.1
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Test failure:
> {{ITestS3AFileContextMainOperations#testRenameDirectoryAsNonExistentDirectory}}
> This is repeatable on -Dauth runs (we haven't been running them, have we?)
> Either its from the recent dir marker changes (initial hypothesis) or its
> been lurking a while and not been picked up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2310: HADOOP-17244. S3A directory delete tombstones dir markers prematurely
steveloughran commented on pull request #2310:
URL: https://github.com/apache/hadoop/pull/2310#issuecomment-714475083
Checkstyle still fussy. What a pain
```
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:93:
"Bulk delete operation failed to delete all objects; failure count =
{}",: Line is longer than 80 characters (found 81). [LineLength]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:331:
List undeleted = extractUndeletedKeyPaths(deleteException,
qualifier);: Line is longer than 80 characters (found 83). [LineLength]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:360:
public final String key;:25: Variable 'key' must be private and have
accessor methods. [VisibilityModifier]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:361:
public final Path path;:23: Variable 'path' must be private and have
accessor methods. [VisibilityModifier]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:362:
public final boolean directoryMarker;:26: Variable 'directoryMarker' must
be private and have accessor methods. [VisibilityModifier]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:398:
if (this == o) { return true; }:22: '{' at column 22 should have line
break after. [LeftCurly]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java:399:
if (o == null || getClass() != o.getClass()) { return false; }:52: '{' at
column 52 should have line break after. [LeftCurly]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestPartialRenamesDeletes.java:668:
MetricDiff deleteObjectCount = new MetricDiff(roleFS,
OBJECT_DELETE_OBJECTS);: Line is longer than 80 characters (found 81).
[LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestPartialRenamesDeletes.java:676:
deleteObjectCount.assertDiffEquals("Wrong count of objects in delete
request",: Line is longer than 80 characters (found 84). [LineLength]
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2401: YARN-10469. YARN-UI2 The accuracy of the percentage values in the same chart on the YARN 'Cluster OverView' page are inconsistent
hadoop-yetus commented on pull request #2401:
URL: https://github.com/apache/hadoop/pull/2401#issuecomment-714445356
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | jshint | 0m 0s | | jshint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 23s | | trunk passed |
| +1 :green_heart: | shadedclient | 46m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 13s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 48s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 64m 16s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2401/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2401 |
| JIRA Issue | YARN-10469 |
| Optional Tests | dupname asflicense shadedclient jshint |
| uname | Linux 88b0502f0282 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7435604a91a |
| Max. process+thread count | 414 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2401/1/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?focusedWorklogId=503675=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503675
]
ASF GitHub Bot logged work on HADOOP-17175:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 11:17
Start Date: 22/Oct/20 11:17
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714423223
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 38s | | trunk passed |
| +1 :green_heart: | compile | 27m 2s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 22m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 59s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 44s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 41s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 24m 40s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 24m 40s | | the patch passed |
| +1 :green_heart: | compile | 17m 28s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 28s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 29s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 3s | | patch has no errors
when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 56s |
[/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/4/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
|
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 99 new + 0
unchanged - 0 fixed = 99 total (was 0) |
| +1 :green_heart: | javadoc | 1m 32s | |
hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new
+ 0 unchanged - 1 fixed = 0 total (was 1) |
| +1 :green_heart: | findbugs | 2m 22s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 9m 41s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/4/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 195m 27s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.ha.TestZKFailoverController |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2397 |
| Optional Tests | dupname
[GitHub] [hadoop] hadoop-yetus commented on pull request #2397: HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module.
hadoop-yetus commented on pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714423223
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 38s | | trunk passed |
| +1 :green_heart: | compile | 27m 2s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 22m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 59s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 44s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 41s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 24m 40s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 24m 40s | | the patch passed |
| +1 :green_heart: | compile | 17m 28s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 28s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 29s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 15m 3s | | patch has no errors
when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 56s |
[/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/4/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
|
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 99 new + 0
unchanged - 0 fixed = 99 total (was 0) |
| +1 :green_heart: | javadoc | 1m 32s | |
hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new
+ 0 unchanged - 1 fixed = 0 total (was 1) |
| +1 :green_heart: | findbugs | 2m 22s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 9m 41s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/4/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 195m 27s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.ha.TestZKFailoverController |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2397 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 947f8ace82cd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7435604a91a |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK
[GitHub] [hadoop] akiyamaneko commented on pull request #2401: YARN-10469. YARN-UI2 The accuracy of the percentage values in the same chart on the YARN 'Cluster OverView' page are inconsistent
akiyamaneko commented on pull request #2401: URL: https://github.com/apache/hadoop/pull/2401#issuecomment-714415749 cc @sunilgovind could you help to review it? thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] akiyamaneko opened a new pull request #2401: [YARN-10469][YARN-UI2] The accuracy of the percentage values in the same chart on the YARN 'Cluster OverView' page are inconsistent
akiyamaneko opened a new pull request #2401: URL: https://github.com/apache/hadoop/pull/2401 Ensure the accuracy of the percentage values in the same chart on the YARN 'Cluster OverView' page are consistent just a small improvement The manual test results are as follows:  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17265) ABFS: Support for Client Correlation ID
[ https://issues.apache.org/jira/browse/HADOOP-17265?focusedWorklogId=503664=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503664 ] ASF GitHub Bot logged work on HADOOP-17265: --- Author: ASF GitHub Bot Created on: 22/Oct/20 10:30 Start Date: 22/Oct/20 10:30 Worklog Time Spent: 10m Work Description: sumangala-patki commented on pull request #2344: URL: https://github.com/apache/hadoop/pull/2344#issuecomment-714398568 The yetus run error is due to tests failing on the WASB side, which is not affected by the changes proposed here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 503664) Time Spent: 2h 20m (was: 2h 10m) > ABFS: Support for Client Correlation ID > --- > > Key: HADOOP-17265 > URL: https://issues.apache.org/jira/browse/HADOOP-17265 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sumangala Patki >Priority: Major > Labels: abfsactive, pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Introducing a client correlation ID that appears in the Azure Storage > diagnostic logs. This will modify the client request header to include the > client-provided identifier for correlating requests. A valid correlation ID > comprises alphanumeric characters and/or hyphens, and can be up to 72 > characters in length. Invalid or missing entries for this configuration are > substituted with the empty string default. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] sumangala-patki commented on pull request #2344: HADOOP-17265. ABFS: Support for Client Correlation ID
sumangala-patki commented on pull request #2344: URL: https://github.com/apache/hadoop/pull/2344#issuecomment-714398568 The yetus run error is due to tests failing on the WASB side, which is not affected by the changes proposed here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2368: Hadoop-17296. ABFS: Force reads to be always of buffer size
hadoop-yetus commented on pull request #2368: URL: https://github.com/apache/hadoop/pull/2368#issuecomment-714389291 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 0s | | Docker mode activated. | | -1 :x: | patch | 0m 4s | | https://github.com/apache/hadoop/pull/2368 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/2368 | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2368/4/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2309: HDFS-15580. [JDK 12] DFSTestUtil#addDataNodeLayoutVersion fails
aajisaka commented on pull request #2309: URL: https://github.com/apache/hadoop/pull/2309#issuecomment-714385828 Rebased. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2377: HDFS-15624. fix the function of setting quota by storage type
hadoop-yetus commented on pull request #2377:
URL: https://github.com/apache/hadoop/pull/2377#issuecomment-714363648
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 33m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 11m 38s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 38s | | trunk passed |
| +1 :green_heart: | compile | 20m 0s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 17m 11s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 13s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 38s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 26s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 26s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 6m 56s | | trunk passed |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 36s | | the patch passed |
| +1 :green_heart: | compile | 19m 14s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 19m 14s | | the patch passed |
| +1 :green_heart: | compile | 17m 9s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 9s | | the patch passed |
| +1 :green_heart: | checkstyle | 2m 50s | | root: The patch generated
0 new + 734 unchanged - 1 fixed = 734 total (was 735) |
| +1 :green_heart: | mvnsite | 3m 55s | | the patch passed |
| -1 :x: | whitespace | 0m 0s |
[/whitespace-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/9/artifact/out/whitespace-eol.txt)
| The patch has 1 line(s) that end in whitespace. Use git apply
--whitespace=fix <>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | shadedclient | 15m 0s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 35s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 4m 23s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 7m 17s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 10m 10s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 124m 28s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/9/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| -1 :x: | unit | 9m 45s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/9/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 1m 6s | | The patch does not
generate ASF License warnings. |
| | | 365m 57s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2377 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d6f94b366d20 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven
[jira] [Work logged] (HADOOP-17319) Update the checkstyle config to ban some guava functions
[
https://issues.apache.org/jira/browse/HADOOP-17319?focusedWorklogId=503648=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503648
]
ASF GitHub Bot logged work on HADOOP-17319:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 09:30
Start Date: 22/Oct/20 09:30
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2400:
URL: https://github.com/apache/hadoop/pull/2400#issuecomment-714363079
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 23s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | mvnsite | 0m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 46m 16s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 14s | | the patch passed |
| +1 :green_heart: | compile | 0m 13s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 13s | | the patch passed |
| +1 :green_heart: | compile | 0m 14s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 13s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 36s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 20s | | hadoop-build-tools in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 67m 37s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2400/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2400 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml |
| uname | Linux 0df38743b41f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7435604a91a |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2400/1/testReport/ |
| Max. process+thread count | 421 (vs. ulimit of 5500) |
| modules | C: hadoop-build-tools U: hadoop-build-tools |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2400/1/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message,
[GitHub] [hadoop] hadoop-yetus commented on pull request #2400: HADOOP-17319. Update the checkstyle config to ban some guava functions.
hadoop-yetus commented on pull request #2400:
URL: https://github.com/apache/hadoop/pull/2400#issuecomment-714363079
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 23s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | mvnsite | 0m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 46m 16s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 14s | | the patch passed |
| +1 :green_heart: | compile | 0m 13s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 13s | | the patch passed |
| +1 :green_heart: | compile | 0m 14s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 13s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 14m 36s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 20s | | hadoop-build-tools in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 67m 37s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2400/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2400 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml |
| uname | Linux 0df38743b41f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7435604a91a |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2400/1/testReport/ |
| Max. process+thread count | 421 (vs. ulimit of 5500) |
| modules | C: hadoop-build-tools U: hadoop-build-tools |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2400/1/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?focusedWorklogId=503641=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503641
]
ASF GitHub Bot logged work on HADOOP-17175:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 09:14
Start Date: 22/Oct/20 09:14
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714353285
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 12s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 31s | | trunk passed |
| +1 :green_heart: | compile | 21m 26s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 18m 15s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 47s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 52s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 34s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 25s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 23s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | | the patch passed |
| +1 :green_heart: | compile | 22m 54s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 22m 54s | | the patch passed |
| +1 :green_heart: | compile | 23m 20s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 23m 20s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 19m 53s | | patch has no errors
when building and testing our client artifacts. |
| -1 :x: | javadoc | 1m 7s |
[/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/3/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
|
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 99 new + 0
unchanged - 0 fixed = 99 total (was 0) |
| +1 :green_heart: | javadoc | 1m 42s | |
hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new
+ 0 unchanged - 1 fixed = 0 total (was 1) |
| +1 :green_heart: | findbugs | 3m 3s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 12m 26s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 191m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2397 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 329f7391a6f3 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[GitHub] [hadoop] YaYun-Wang removed a comment on pull request #2377: HDFS-15624. fix the function of setting quota by storage type
YaYun-Wang removed a comment on pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#issuecomment-714352736 ok, we will fix that all in the pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] YaYun-Wang commented on pull request #2377: HDFS-15624. fix the function of setting quota by storage type
YaYun-Wang commented on pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#issuecomment-714353591 @ayushtkn ,ok, we will fix that all in the pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2397: HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module.
hadoop-yetus commented on pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714353285
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 12s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 31s | | trunk passed |
| +1 :green_heart: | compile | 21m 26s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 18m 15s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 47s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 52s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 34s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 25s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 23s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | | the patch passed |
| +1 :green_heart: | compile | 22m 54s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 22m 54s | | the patch passed |
| +1 :green_heart: | compile | 23m 20s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 23m 20s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 19m 53s | | patch has no errors
when building and testing our client artifacts. |
| -1 :x: | javadoc | 1m 7s |
[/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/3/artifact/out/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt)
|
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 99 new + 0
unchanged - 0 fixed = 99 total (was 0) |
| +1 :green_heart: | javadoc | 1m 42s | |
hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 0 new
+ 0 unchanged - 1 fixed = 0 total (was 1) |
| +1 :green_heart: | findbugs | 3m 3s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 12m 26s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 191m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2397 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 329f7391a6f3 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7f8ef76c483 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2397/3/testReport/ |
|
[GitHub] [hadoop] YaYun-Wang commented on pull request #2377: HDFS-15624. fix the function of setting quota by storage type
YaYun-Wang commented on pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#issuecomment-714352736 ok, we will fix that all in the pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2377: HDFS-15624. fix the function of setting quota by storage type
hadoop-yetus commented on pull request #2377:
URL: https://github.com/apache/hadoop/pull/2377#issuecomment-714349635
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 11m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 23m 9s | | trunk passed |
| +1 :green_heart: | compile | 21m 26s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 18m 5s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 55s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 42s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 14s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 58s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 1m 19s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 6m 51s | | trunk passed |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 38s | | the patch passed |
| +1 :green_heart: | compile | 20m 51s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 20m 51s | | the patch passed |
| +1 :green_heart: | compile | 18m 0s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 18m 0s | | the patch passed |
| +1 :green_heart: | checkstyle | 2m 56s | | root: The patch generated
0 new + 734 unchanged - 1 fixed = 734 total (was 735) |
| +1 :green_heart: | mvnsite | 3m 33s | | the patch passed |
| -1 :x: | whitespace | 0m 0s |
[/whitespace-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/8/artifact/out/whitespace-eol.txt)
| The patch has 1 line(s) that end in whitespace. Use git apply
--whitespace=fix <>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | shadedclient | 16m 38s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 13s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 3m 58s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 7m 15s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 9m 34s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 128m 49s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/8/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | unit | 10m 20s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 344m 43s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestMiniDFSCluster |
| | hadoop.hdfs.TestDFSStripedOutputStream |
| | hadoop.hdfs.TestReadStripedFileWithDNFailure |
| | hadoop.hdfs.TestFileChecksumCompositeCrc |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.security.TestDelegationToken |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2377/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2377 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d82bc2f669bd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7f8ef76c483 |
| Default Java
[GitHub] [hadoop] hadoop-yetus commented on pull request #2309: HDFS-15580. [JDK 12] DFSTestUtil#addDataNodeLayoutVersion fails
hadoop-yetus commented on pull request #2309: URL: https://github.com/apache/hadoop/pull/2309#issuecomment-714328504 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 0s | | Docker mode activated. | | -1 :x: | patch | 0m 5s | | https://github.com/apache/hadoop/pull/2309 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/2309 | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2309/5/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?focusedWorklogId=503619=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503619
]
ASF GitHub Bot logged work on HADOOP-17175:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 08:31
Start Date: 22/Oct/20 08:31
Worklog Time Spent: 10m
Work Description: tasanuma commented on pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714327287
@aajisaka Thanks for updating the PR. +1, pending Jenkins.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503619)
Time Spent: 1.5h (was: 1h 20m)
> [JDK11] Fix javadoc errors in hadoop-common module
> --
>
> Key: HADOOP-17175
> URL: https://issues.apache.org/jira/browse/HADOOP-17175
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Full error log:
> https://gist.github.com/aajisaka/5504dd485cc4bc98a6a245830d450919
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java:57:
> error: unexpected end tag:
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedTimeCostProvider.java:33:
> error: self-closing element not allowed
> [ERROR] * This allows for configuration of how heavily each of the
> operations
> [ERROR]^
> ...
> {noformat}
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tasanuma commented on pull request #2397: HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module.
tasanuma commented on pull request #2397: URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714327287 @aajisaka Thanks for updating the PR. +1, pending Jenkins. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17319) Update the checkstyle config to ban some guava functions
[ https://issues.apache.org/jira/browse/HADOOP-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17319: --- Parent: HADOOP-17098 Issue Type: Sub-task (was: Bug) > Update the checkstyle config to ban some guava functions > > > Key: HADOOP-17319 > URL: https://issues.apache.org/jira/browse/HADOOP-17319 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Some guava functions are banned in HADOOP-17111, HADOOP-17099, and > HADOOP-17101 via checkstyle, however, the checkstyle configuration does not > work after HADOOP-17288 because the package names have been changed. > Originally reported by [~ahussein] in HADOOP-17315. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17319) Update the checkstyle config to ban some guava functions
[ https://issues.apache.org/jira/browse/HADOOP-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17319: Labels: pull-request-available (was: ) > Update the checkstyle config to ban some guava functions > > > Key: HADOOP-17319 > URL: https://issues.apache.org/jira/browse/HADOOP-17319 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Some guava functions are banned in HADOOP-17111, HADOOP-17099, and > HADOOP-17101 via checkstyle, however, the checkstyle configuration does not > work after HADOOP-17288 because the package names have been changed. > Originally reported by [~ahussein] in HADOOP-17315. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17319) Update the checkstyle config to ban some guava functions
[ https://issues.apache.org/jira/browse/HADOOP-17319?focusedWorklogId=503616=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503616 ] ASF GitHub Bot logged work on HADOOP-17319: --- Author: ASF GitHub Bot Created on: 22/Oct/20 08:21 Start Date: 22/Oct/20 08:21 Worklog Time Spent: 10m Work Description: aajisaka opened a new pull request #2400: URL: https://github.com/apache/hadoop/pull/2400 JIRA: https://issues.apache.org/jira/browse/HADOOP-17319 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 503616) Remaining Estimate: 0h Time Spent: 10m > Update the checkstyle config to ban some guava functions > > > Key: HADOOP-17319 > URL: https://issues.apache.org/jira/browse/HADOOP-17319 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Some guava functions are banned in HADOOP-17111, HADOOP-17099, and > HADOOP-17101 via checkstyle, however, the checkstyle configuration does not > work after HADOOP-17288 because the package names have been changed. > Originally reported by [~ahussein] in HADOOP-17315. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka opened a new pull request #2400: HADOOP-17319. Update the checkstyle config to ban some guava functions.
aajisaka opened a new pull request #2400: URL: https://github.com/apache/hadoop/pull/2400 JIRA: https://issues.apache.org/jira/browse/HADOOP-17319 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17319) Update the checkstyle config to ban some guava functions
[ https://issues.apache.org/jira/browse/HADOOP-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-17319: -- Assignee: Akira Ajisaka > Update the checkstyle config to ban some guava functions > > > Key: HADOOP-17319 > URL: https://issues.apache.org/jira/browse/HADOOP-17319 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > > Some guava functions are banned in HADOOP-17111, HADOOP-17099, and > HADOOP-17101 via checkstyle, however, the checkstyle configuration does not > work after HADOOP-17288 because the package names have been changed. > Originally reported by [~ahussein] in HADOOP-17315. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?focusedWorklogId=503609=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503609
]
ASF GitHub Bot logged work on HADOOP-17175:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 08:01
Start Date: 22/Oct/20 08:01
Worklog Time Spent: 10m
Work Description: aajisaka commented on a change in pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#discussion_r509957549
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ClientCache.java
##
@@ -29,7 +29,7 @@
import org.apache.hadoop.io.ObjectWritable;
import org.apache.hadoop.io.Writable;
-import
org.apache.hadoop.thirdparty.com.google.common.annotations.VisibleForTesting;
+import com.google.common.annotations.VisibleForTesting;
Review comment:
Yes, it was a mistake. Fixed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503609)
Time Spent: 1h 20m (was: 1h 10m)
> [JDK11] Fix javadoc errors in hadoop-common module
> --
>
> Key: HADOOP-17175
> URL: https://issues.apache.org/jira/browse/HADOOP-17175
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Full error log:
> https://gist.github.com/aajisaka/5504dd485cc4bc98a6a245830d450919
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java:57:
> error: unexpected end tag:
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedTimeCostProvider.java:33:
> error: self-closing element not allowed
> [ERROR] * This allows for configuration of how heavily each of the
> operations
> [ERROR]^
> ...
> {noformat}
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #2397: HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module.
aajisaka commented on a change in pull request #2397: URL: https://github.com/apache/hadoop/pull/2397#discussion_r509957549 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ClientCache.java ## @@ -29,7 +29,7 @@ import org.apache.hadoop.io.ObjectWritable; import org.apache.hadoop.io.Writable; -import org.apache.hadoop.thirdparty.com.google.common.annotations.VisibleForTesting; +import com.google.common.annotations.VisibleForTesting; Review comment: Yes, it was a mistake. Fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?focusedWorklogId=503588=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503588
]
ASF GitHub Bot logged work on HADOOP-17175:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 07:25
Start Date: 22/Oct/20 07:25
Worklog Time Spent: 10m
Work Description: tasanuma commented on a change in pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#discussion_r509936009
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ClientCache.java
##
@@ -29,7 +29,7 @@
import org.apache.hadoop.io.ObjectWritable;
import org.apache.hadoop.io.Writable;
-import
org.apache.hadoop.thirdparty.com.google.common.annotations.VisibleForTesting;
+import com.google.common.annotations.VisibleForTesting;
Review comment:
Is this a mistake?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503588)
Time Spent: 1h 10m (was: 1h)
> [JDK11] Fix javadoc errors in hadoop-common module
> --
>
> Key: HADOOP-17175
> URL: https://issues.apache.org/jira/browse/HADOOP-17175
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Full error log:
> https://gist.github.com/aajisaka/5504dd485cc4bc98a6a245830d450919
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java:57:
> error: unexpected end tag:
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedTimeCostProvider.java:33:
> error: self-closing element not allowed
> [ERROR] * This allows for configuration of how heavily each of the
> operations
> [ERROR]^
> ...
> {noformat}
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tasanuma commented on a change in pull request #2397: HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module.
tasanuma commented on a change in pull request #2397: URL: https://github.com/apache/hadoop/pull/2397#discussion_r509936009 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ClientCache.java ## @@ -29,7 +29,7 @@ import org.apache.hadoop.io.ObjectWritable; import org.apache.hadoop.io.Writable; -import org.apache.hadoop.thirdparty.com.google.common.annotations.VisibleForTesting; +import com.google.common.annotations.VisibleForTesting; Review comment: Is this a mistake? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2377: HDFS-15624. fix the function of setting quota by storage type
liuml07 commented on pull request #2377: URL: https://github.com/apache/hadoop/pull/2377#issuecomment-714277499 Thanks @YaYun-Wang for fixing this. It looks good to me overall, and I can have another look over the weekend, if it's not committed. I also added @ayushtkn as reviewer. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17319) Update the checkstyle config to ban some guava functions
[ https://issues.apache.org/jira/browse/HADOOP-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218794#comment-17218794 ] Akira Ajisaka commented on HADOOP-17319: Thanks [~ahussein] for your comment. bq. For example we can call it "TestingScope". VisibleForTesting is very famous inside and outside Apache Hadoop and I think the name should not be changed for now. If we rename this, it may be surprising for the contributors. bq. Otherwise, VisibltForTesting does not bring anything to the table but loading bunch of irrelevant classes such as GwtCompatible, Documented, ..etc. Did anyone consider doing that for the hadoop repository? Now GwtCompatible and Beta annotations are used in org.apache.hadoop.hdfs.server.datanode.checker.AbstractChecker, so I didn't consider doing that. > Update the checkstyle config to ban some guava functions > > > Key: HADOOP-17319 > URL: https://issues.apache.org/jira/browse/HADOOP-17319 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Akira Ajisaka >Priority: Major > > Some guava functions are banned in HADOOP-17111, HADOOP-17099, and > HADOOP-17101 via checkstyle, however, the checkstyle configuration does not > work after HADOOP-17288 because the package names have been changed. > Originally reported by [~ahussein] in HADOOP-17315. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17175) [JDK11] Fix javadoc errors in hadoop-common module
[
https://issues.apache.org/jira/browse/HADOOP-17175?focusedWorklogId=503561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503561
]
ASF GitHub Bot logged work on HADOOP-17175:
---
Author: ASF GitHub Bot
Created on: 22/Oct/20 06:02
Start Date: 22/Oct/20 06:02
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #2397:
URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714251109
Rebased. Thank you @tasanuma
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503561)
Time Spent: 1h (was: 50m)
> [JDK11] Fix javadoc errors in hadoop-common module
> --
>
> Key: HADOOP-17175
> URL: https://issues.apache.org/jira/browse/HADOOP-17175
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Full error log:
> https://gist.github.com/aajisaka/5504dd485cc4bc98a6a245830d450919
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java:57:
> error: unexpected end tag:
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedTimeCostProvider.java:33:
> error: self-closing element not allowed
> [ERROR] * This allows for configuration of how heavily each of the
> operations
> [ERROR]^
> ...
> {noformat}
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2397: HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module.
aajisaka commented on pull request #2397: URL: https://github.com/apache/hadoop/pull/2397#issuecomment-714251109 Rebased. Thank you @tasanuma This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
