[jira] [Updated] (MAPREDUCE-7348) TestFrameworkUploader#testNativeIO fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated MAPREDUCE-7348:
-
Status: Patch Available (was: Open)
> TestFrameworkUploader#testNativeIO fails
>
>
> Key: MAPREDUCE-7348
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7348
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TestFrameworkUploader#testNativeIO fails.
> {noformat}
> [INFO] Running org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] Tests run: 16, Failures: 0, Errors: 3, Skipped: 0, Time elapsed:
> 8.038 s <<< FAILURE! - in
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] testNativeIO(org.apache.hadoop.mapred.uploader.TestFrameworkUploader)
> Time elapsed: 0.026 s <<< ERROR!
> org.apache.commons.io.IOExceptionList: 2 exceptions: [java.io.IOException:
> Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget2.txt,
> java.io.IOException: Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget.txt]
> at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:345)
> at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1206)
> at
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader.testNativeIO(TestFrameworkUploader.java:480)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/518/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Work logged] (MAPREDUCE-7348) TestFrameworkUploader#testNativeIO fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7348?focusedWorklogId=602102=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602102
]
ASF GitHub Bot logged work on MAPREDUCE-7348:
-
Author: ASF GitHub Bot
Created on: 26/May/21 05:04
Start Date: 26/May/21 05:04
Worklog Time Spent: 10m
Work Description: aajisaka opened a new pull request #3053:
URL: https://github.com/apache/hadoop/pull/3053
JIRA: MAPREDUCE-7348.
The behavior of `FileUtils#deleteDirectory` is still changed even in
commons-io 2.8. Use `FileUtils#forceDelete` instead of
`FileUtils#deleteDirectory` to clean up the test dir.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602102)
Remaining Estimate: 0h
Time Spent: 10m
> TestFrameworkUploader#testNativeIO fails
>
>
> Key: MAPREDUCE-7348
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7348
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TestFrameworkUploader#testNativeIO fails.
> {noformat}
> [INFO] Running org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] Tests run: 16, Failures: 0, Errors: 3, Skipped: 0, Time elapsed:
> 8.038 s <<< FAILURE! - in
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] testNativeIO(org.apache.hadoop.mapred.uploader.TestFrameworkUploader)
> Time elapsed: 0.026 s <<< ERROR!
> org.apache.commons.io.IOExceptionList: 2 exceptions: [java.io.IOException:
> Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget2.txt,
> java.io.IOException: Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget.txt]
> at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:345)
> at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1206)
> at
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader.testNativeIO(TestFrameworkUploader.java:480)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/518/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7348) TestFrameworkUploader#testNativeIO fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated MAPREDUCE-7348:
--
Labels: pull-request-available (was: )
> TestFrameworkUploader#testNativeIO fails
>
>
> Key: MAPREDUCE-7348
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7348
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TestFrameworkUploader#testNativeIO fails.
> {noformat}
> [INFO] Running org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] Tests run: 16, Failures: 0, Errors: 3, Skipped: 0, Time elapsed:
> 8.038 s <<< FAILURE! - in
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] testNativeIO(org.apache.hadoop.mapred.uploader.TestFrameworkUploader)
> Time elapsed: 0.026 s <<< ERROR!
> org.apache.commons.io.IOExceptionList: 2 exceptions: [java.io.IOException:
> Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget2.txt,
> java.io.IOException: Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget.txt]
> at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:345)
> at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1206)
> at
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader.testNativeIO(TestFrameworkUploader.java:480)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/518/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-7348) TestFrameworkUploader#testNativeIO fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned MAPREDUCE-7348:
Assignee: Akira Ajisaka
> TestFrameworkUploader#testNativeIO fails
>
>
> Key: MAPREDUCE-7348
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7348
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
>
>
> TestFrameworkUploader#testNativeIO fails.
> {noformat}
> [INFO] Running org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] Tests run: 16, Failures: 0, Errors: 3, Skipped: 0, Time elapsed:
> 8.038 s <<< FAILURE! - in
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] testNativeIO(org.apache.hadoop.mapred.uploader.TestFrameworkUploader)
> Time elapsed: 0.026 s <<< ERROR!
> org.apache.commons.io.IOExceptionList: 2 exceptions: [java.io.IOException:
> Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget2.txt,
> java.io.IOException: Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget.txt]
> at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:345)
> at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1206)
> at
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader.testNativeIO(TestFrameworkUploader.java:480)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/518/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7348) TestFrameworkUploader#testNativeIO fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated MAPREDUCE-7348:
-
Target Version/s: 3.4.0, 3.3.1 (was: 3.4.0)
> TestFrameworkUploader#testNativeIO fails
>
>
> Key: MAPREDUCE-7348
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7348
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Priority: Major
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
>
>
> TestFrameworkUploader#testNativeIO fails.
> {noformat}
> [INFO] Running org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] Tests run: 16, Failures: 0, Errors: 3, Skipped: 0, Time elapsed:
> 8.038 s <<< FAILURE! - in
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] testNativeIO(org.apache.hadoop.mapred.uploader.TestFrameworkUploader)
> Time elapsed: 0.026 s <<< ERROR!
> org.apache.commons.io.IOExceptionList: 2 exceptions: [java.io.IOException:
> Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget2.txt,
> java.io.IOException: Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget.txt]
> at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:345)
> at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1206)
> at
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader.testNativeIO(TestFrameworkUploader.java:480)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/518/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7348) TestFrameworkUploader#testNativeIO fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated MAPREDUCE-7348:
-
Attachment:
patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
> TestFrameworkUploader#testNativeIO fails
>
>
> Key: MAPREDUCE-7348
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7348
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Priority: Major
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
>
>
> TestFrameworkUploader#testNativeIO fails.
> {noformat}
> [INFO] Running org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] Tests run: 16, Failures: 0, Errors: 3, Skipped: 0, Time elapsed:
> 8.038 s <<< FAILURE! - in
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader
> [ERROR] testNativeIO(org.apache.hadoop.mapred.uploader.TestFrameworkUploader)
> Time elapsed: 0.026 s <<< ERROR!
> org.apache.commons.io.IOExceptionList: 2 exceptions: [java.io.IOException:
> Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget2.txt,
> java.io.IOException: Unable to delete file:
> /home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget.txt]
> at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:345)
> at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1206)
> at
> org.apache.hadoop.mapred.uploader.TestFrameworkUploader.testNativeIO(TestFrameworkUploader.java:480)
> {noformat}
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/518/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7348) TestFrameworkUploader#testNativeIO fails
Akira Ajisaka created MAPREDUCE-7348:
Summary: TestFrameworkUploader#testNativeIO fails
Key: MAPREDUCE-7348
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7348
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Reporter: Akira Ajisaka
Attachments:
patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
TestFrameworkUploader#testNativeIO fails.
{noformat}
[INFO] Running org.apache.hadoop.mapred.uploader.TestFrameworkUploader
[ERROR] Tests run: 16, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 8.038
s <<< FAILURE! - in org.apache.hadoop.mapred.uploader.TestFrameworkUploader
[ERROR] testNativeIO(org.apache.hadoop.mapred.uploader.TestFrameworkUploader)
Time elapsed: 0.026 s <<< ERROR!
org.apache.commons.io.IOExceptionList: 2 exceptions: [java.io.IOException:
Unable to delete file:
/home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget2.txt,
java.io.IOException: Unable to delete file:
/home/jenkins/jenkins-home/workspace/hadoop-qbt-trunk-java8-linux-x86_64/sourcedir/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/target/test-dir/3575932722382085229/symlinkToTarget.txt]
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:345)
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1206)
at
org.apache.hadoop.mapred.uploader.TestFrameworkUploader.testNativeIO(TestFrameworkUploader.java:480)
{noformat}
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/518/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Work logged] (MAPREDUCE-7341) Add a task-manifest output committer for Azure and GCS
[
https://issues.apache.org/jira/browse/MAPREDUCE-7341?focusedWorklogId=602014=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602014
]
ASF GitHub Bot logged work on MAPREDUCE-7341:
-
Author: ASF GitHub Bot
Created on: 25/May/21 23:19
Start Date: 25/May/21 23:19
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2971:
URL: https://github.com/apache/hadoop/pull/2971#issuecomment-848333030
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 17 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 1s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 16s | | trunk passed |
| +1 :green_heart: | compile | 20m 53s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 18m 13s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 47s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 13s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 40s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 40s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 14m 18s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 14m 42s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 14s | | the patch passed |
| +1 :green_heart: | compile | 20m 25s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 20m 25s |
[/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/11/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 generated 2 new + 1994 unchanged - 1
fixed = 1996 total (was 1995) |
| +1 :green_heart: | compile | 18m 11s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| -1 :x: | javac | 18m 11s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/11/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
| root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 2 new + 1867
unchanged - 1 fixed = 1869 total (was 1868) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/11/artifact/out/blanks-eol.txt)
| The patch has 6 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 3m 43s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/11/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 58 new + 0 unchanged - 0 fixed = 58 total (was 0)
|
| +1 :green_heart: | mvnsite | 4m 6s | | the patch passed |
| +1 :green_heart: | xml | 0m 8s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 3m 12s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 46s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 39s | | hadoop-project has no data from
spotbugs |
| -1 :x: | spotbugs | 1m 45s |
[jira] [Work logged] (MAPREDUCE-7341) Add a task-manifest output committer for Azure and GCS
[
https://issues.apache.org/jira/browse/MAPREDUCE-7341?focusedWorklogId=601909=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-601909
]
ASF GitHub Bot logged work on MAPREDUCE-7341:
-
Author: ASF GitHub Bot
Created on: 25/May/21 18:51
Start Date: 25/May/21 18:51
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2971:
URL: https://github.com/apache/hadoop/pull/2971#issuecomment-848166747
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 17 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 25s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 1s | | trunk passed |
| +1 :green_heart: | compile | 20m 47s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 18m 10s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 13s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 37s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 42s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 14m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 13s | | the patch passed |
| +1 :green_heart: | compile | 20m 30s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 20m 30s |
[/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/10/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 generated 2 new + 1994 unchanged - 1
fixed = 1996 total (was 1995) |
| +1 :green_heart: | compile | 18m 9s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| -1 :x: | javac | 18m 9s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/10/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
| root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 2 new + 1867
unchanged - 1 fixed = 1869 total (was 1868) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/10/artifact/out/blanks-eol.txt)
| The patch has 5 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 3m 43s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/10/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 57 new + 0 unchanged - 0 fixed = 57 total (was 0)
|
| +1 :green_heart: | mvnsite | 4m 5s | | the patch passed |
| +1 :green_heart: | xml | 0m 8s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 3m 13s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 39s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 38s | | hadoop-project has no data from
spotbugs |
| -1 :x: | spotbugs | 1m 45s |
[jira] [Work logged] (MAPREDUCE-7339) Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
[
https://issues.apache.org/jira/browse/MAPREDUCE-7339?focusedWorklogId=601742=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-601742
]
ASF GitHub Bot logged work on MAPREDUCE-7339:
-
Author: ASF GitHub Bot
Created on: 25/May/21 14:02
Start Date: 25/May/21 14:02
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2951:
URL: https://github.com/apache/hadoop/pull/2951#issuecomment-847896744
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 14s | | trunk passed |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 35s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2951/1/artifact/out/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt)
|
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:
The patch generated 2 new + 51 unchanged - 0 fixed = 53 total (was 51) |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 16s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 8m 15s | | hadoop-mapreduce-client-app in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 84m 23s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2951/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2951 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 6fbb44e26648 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13
01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4df9782c1d706327e928596f3830a80c0a251937 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
[jira] [Commented] (MAPREDUCE-7347) Add a Hive platform to Nativetask
[
https://issues.apache.org/jira/browse/MAPREDUCE-7347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351070#comment-17351070
]
Hadoop QA commented on MAPREDUCE-7347:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m
34s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to
include any new or modified tests. Please justify why no new tests are needed
for this patch. Also please list what manual steps were performed to verify
this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
31s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
33s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 45s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 20m
20s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 0m
42s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 1m 7s{color} |
{color:red}https://ci-hadoop.apache.org/job/PreCommit-MAPREDUCE-Build/77/artifact/out/diff-compile-cc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt{color}
| {color:red}
hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 27 new + 93
unchanged - 27 fixed = 120 total (was 120) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 1m
7s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 1m 8s{color} |
[jira] [Updated] (MAPREDUCE-7347) Add a Hive platform to Nativetask
[ https://issues.apache.org/jira/browse/MAPREDUCE-7347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jiacheng updated MAPREDUCE-7347: - Description: For now, Native task can not be used by Hive job, because we need to register the key. And as we all know, HiveKey, which is the key in Hive job, is a child class of BytesWritable. So all we need to do is just to register the HiveKey as BytesWritable. This solution has already used online in our cluster, we got tens of thousands sqls submitted to the cluster. was: For now, Native task can not be used by Hive job, because we need to register the key. And as we all know, HiveKey, which is the key in Hive job, is a child class of BytesWritable. So what we need to do is just register the HiveKey as BytesWritable. This solution has already used online in our cluster, we got tens of thousands sqls submitted to the cluster. > Add a Hive platform to Nativetask > - > > Key: MAPREDUCE-7347 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7347 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: nativetask >Affects Versions: 3.2.1 >Reporter: Song Jiacheng >Priority: Major > Attachments: MAPREDUCE-7347.v1.patch > > > For now, Native task can not be used by Hive job, because we need to register > the key. > And as we all know, HiveKey, which is the key in Hive job, is a child class > of BytesWritable. > So all we need to do is just to register the HiveKey as BytesWritable. > This solution has already used online in our cluster, we got tens of > thousands sqls submitted to the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7347) Add a Hive platform to Nativetask
[ https://issues.apache.org/jira/browse/MAPREDUCE-7347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jiacheng updated MAPREDUCE-7347: - Attachment: MAPREDUCE-7347.v1.patch Status: Patch Available (was: Open) > Add a Hive platform to Nativetask > - > > Key: MAPREDUCE-7347 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7347 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: nativetask >Affects Versions: 3.2.1 >Reporter: Song Jiacheng >Priority: Major > Attachments: MAPREDUCE-7347.v1.patch > > > For now, Native task can not be used by Hive job, because we need to register > the key. > And as we all know, HiveKey, which is the key in Hive job, is a child class > of BytesWritable. > So what we need to do is just register the HiveKey as BytesWritable. > This solution has already used online in our cluster, we got tens of > thousands sqls submitted to the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7347) Add a Hive platform to Nativetask
[ https://issues.apache.org/jira/browse/MAPREDUCE-7347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jiacheng updated MAPREDUCE-7347: - Component/s: nativetask > Add a Hive platform to Nativetask > - > > Key: MAPREDUCE-7347 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7347 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: nativetask >Affects Versions: 3.2.1 >Reporter: Song Jiacheng >Priority: Major > > For now, Native task can not be used by Hive job, because we need to register > the key. > And as we all know, HiveKey, which is the key in Hive job, is a child class > of BytesWritable. > So what we need to do is just register the HiveKey as BytesWritable. > This solution has already used online in our cluster, we got tens of > thousands sqls submitted to the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7347) Add a Hive platform to Nativetask
Song Jiacheng created MAPREDUCE-7347: Summary: Add a Hive platform to Nativetask Key: MAPREDUCE-7347 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7347 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.2.1 Reporter: Song Jiacheng For now, Native task can not be used by Hive job, because we need to register the key. And as we all know, HiveKey, which is the key in Hive job, is a child class of BytesWritable. So what we need to do is just register the HiveKey as BytesWritable. This solution has already used online in our cluster, we got tens of thousands sqls submitted to the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7347) Add a Hive platform to Nativetask
[ https://issues.apache.org/jira/browse/MAPREDUCE-7347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Song Jiacheng updated MAPREDUCE-7347: - Issue Type: Improvement (was: Bug) > Add a Hive platform to Nativetask > - > > Key: MAPREDUCE-7347 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7347 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 3.2.1 >Reporter: Song Jiacheng >Priority: Major > > For now, Native task can not be used by Hive job, because we need to register > the key. > And as we all know, HiveKey, which is the key in Hive job, is a child class > of BytesWritable. > So what we need to do is just register the HiveKey as BytesWritable. > This solution has already used online in our cluster, we got tens of > thousands sqls submitted to the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Work logged] (MAPREDUCE-7346) [Gridmix] A potential divide by zero in AvgRecordFactory
[
https://issues.apache.org/jira/browse/MAPREDUCE-7346?focusedWorklogId=601626=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-601626
]
ASF GitHub Bot logged work on MAPREDUCE-7346:
-
Author: ASF GitHub Bot
Created on: 25/May/21 10:50
Start Date: 25/May/21 10:50
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3047:
URL: https://github.com/apache/hadoop/pull/3047#issuecomment-847765028
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 25s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 44s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 24s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | compile | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 14s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 49s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 26s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 38s | | hadoop-gridmix in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 94m 24s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3047/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3047 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 57eba8cc8f8b 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 77da40b8bb6f84c78d3a970bb5a71db7e94f462f |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3047/2/testReport/ |
| Max. process+thread count | 979 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-gridmix U: hadoop-tools/hadoop-gridmix |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3047/2/console |
| versions |
[jira] [Work logged] (MAPREDUCE-7346) [Gridmix] A potential divide by zero in AvgRecordFactory
[
https://issues.apache.org/jira/browse/MAPREDUCE-7346?focusedWorklogId=601585=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-601585
]
ASF GitHub Bot logged work on MAPREDUCE-7346:
-
Author: ASF GitHub Bot
Created on: 25/May/21 09:02
Start Date: 25/May/21 09:02
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3047:
URL: https://github.com/apache/hadoop/pull/3047#issuecomment-847690254
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 17m 15s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 5s | | trunk passed |
| +1 :green_heart: | compile | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 25s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 29s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 25s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 23s | | the patch passed |
| +1 :green_heart: | compile | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 22s | | the patch passed |
| +1 :green_heart: | compile | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 19s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3047/1/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3047/1/artifact/out/blanks-tabs.txt)
| The patch 1 line(s) with tabs. |
| -0 :warning: | checkstyle | 0m 14s |
[/results-checkstyle-hadoop-tools_hadoop-gridmix.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3047/1/artifact/out/results-checkstyle-hadoop-tools_hadoop-gridmix.txt)
| hadoop-tools/hadoop-gridmix: The patch generated 1 new + 3 unchanged - 0
fixed = 4 total (was 3) |
| +1 :green_heart: | mvnsite | 0m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 15s | | hadoop-gridmix in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 30s | | The patch does not
generate ASF License warnings. |
| | | 108m 34s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3047/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3047 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux d5641ead0c6e 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
[jira] [Updated] (MAPREDUCE-7346) [Gridmix] A potential divide by zero in AvgRecordFactory
[
https://issues.apache.org/jira/browse/MAPREDUCE-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiyuan GUO updated MAPREDUCE-7346:
--
External issue URL: https://github.com/apache/hadoop/pull/3047
> [Gridmix] A potential divide by zero in AvgRecordFactory
>
>
> Key: MAPREDUCE-7346
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7346
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/gridmix
>Reporter: Yiyuan GUO
>Priority: Major
> Labels: gridmix, pull-request-available, security
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the file _gridmix/AvgRecordFactory.java,_ one of the class's constructor
> has the following
> [code|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/AvgRecordFactory.java#L63-#L68]:
> {code:java}
> public AvgRecordFactory(long targetBytes, long targetRecords,
> Configuration conf, int minSpilledBytes) {
> this.targetBytes = targetBytes;
> this.targetRecords = targetRecords <= 0 && this.targetBytes >= 0
> ? Math.max(1,
> this.targetBytes / conf.getInt(GRIDMIX_MISSING_REC_SIZE, 64 * 1024))
> : targetRecords;
> final long tmp = this.targetBytes / this.targetRecords;
> ...
> }
> {code}
> The check in the code _targetRecords <= 0 && this.targetBytes >= 0_ is not
> sufficient: when _*targetRecords == 0 && targetBytes < 0*,_ we can bypass the
> protection checking and set _this.targetRecords_ to zero, leading to a divide
> by zero problem when computing _tmp._
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7346) [Gridmix] A potential divide by zero in AvgRecordFactory
[
https://issues.apache.org/jira/browse/MAPREDUCE-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated MAPREDUCE-7346:
--
Labels: gridmix pull-request-available security (was: gridmix security)
> [Gridmix] A potential divide by zero in AvgRecordFactory
>
>
> Key: MAPREDUCE-7346
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7346
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/gridmix
>Reporter: Yiyuan GUO
>Priority: Major
> Labels: gridmix, pull-request-available, security
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the file _gridmix/AvgRecordFactory.java,_ one of the class's constructor
> has the following
> [code|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/AvgRecordFactory.java#L63-#L68]:
> {code:java}
> public AvgRecordFactory(long targetBytes, long targetRecords,
> Configuration conf, int minSpilledBytes) {
> this.targetBytes = targetBytes;
> this.targetRecords = targetRecords <= 0 && this.targetBytes >= 0
> ? Math.max(1,
> this.targetBytes / conf.getInt(GRIDMIX_MISSING_REC_SIZE, 64 * 1024))
> : targetRecords;
> final long tmp = this.targetBytes / this.targetRecords;
> ...
> }
> {code}
> The check in the code _targetRecords <= 0 && this.targetBytes >= 0_ is not
> sufficient: when _*targetRecords == 0 && targetBytes < 0*,_ we can bypass the
> protection checking and set _this.targetRecords_ to zero, leading to a divide
> by zero problem when computing _tmp._
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Work logged] (MAPREDUCE-7346) [Gridmix] A potential divide by zero in AvgRecordFactory
[
https://issues.apache.org/jira/browse/MAPREDUCE-7346?focusedWorklogId=601557=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-601557
]
ASF GitHub Bot logged work on MAPREDUCE-7346:
-
Author: ASF GitHub Bot
Created on: 25/May/21 07:12
Start Date: 25/May/21 07:12
Worklog Time Spent: 10m
Work Description: yiyuaner opened a new pull request #3047:
URL: https://github.com/apache/hadoop/pull/3047
This commit fixes the issue
[here](https://issues.apache.org/jira/browse/MAPREDUCE-7346).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 601557)
Remaining Estimate: 0h
Time Spent: 10m
> [Gridmix] A potential divide by zero in AvgRecordFactory
>
>
> Key: MAPREDUCE-7346
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7346
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/gridmix
>Reporter: Yiyuan GUO
>Priority: Major
> Labels: gridmix, security
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the file _gridmix/AvgRecordFactory.java,_ one of the class's constructor
> has the following
> [code|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/AvgRecordFactory.java#L63-#L68]:
> {code:java}
> public AvgRecordFactory(long targetBytes, long targetRecords,
> Configuration conf, int minSpilledBytes) {
> this.targetBytes = targetBytes;
> this.targetRecords = targetRecords <= 0 && this.targetBytes >= 0
> ? Math.max(1,
> this.targetBytes / conf.getInt(GRIDMIX_MISSING_REC_SIZE, 64 * 1024))
> : targetRecords;
> final long tmp = this.targetBytes / this.targetRecords;
> ...
> }
> {code}
> The check in the code _targetRecords <= 0 && this.targetBytes >= 0_ is not
> sufficient: when _*targetRecords == 0 && targetBytes < 0*,_ we can bypass the
> protection checking and set _this.targetRecords_ to zero, leading to a divide
> by zero problem when computing _tmp._
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
