[jira] [Commented] (HADOOP-14178) Move Mockito up to version 2.x
[
https://issues.apache.org/jira/browse/HADOOP-14178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752947#comment-16752947
]
Hadoop QA commented on HADOOP-14178:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 288 new or modified
test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 14m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
32m 8s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-hdfs-project/hadoop-hdfs-native-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project
hadoop-client-modules/hadoop-client-minicluster . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
29s{color} | {color:red} framework in trunk failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
24s{color} | {color:red} server-scm in trunk failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
17s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
30s{color} | {color:red} hadoop-aws in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 20m
38s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
13s{color} | {color:red} framework in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
14s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 14m
13s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 14m 13s{color}
| {color:red} root in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
5m 10s{color} | {color:orange} root: The patch generated 2 new + 6971 unchanged
- 70 fixed = 6973 total (was 7041) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 13m
23s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
55s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 26s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-hdfs-project/hadoop-hdfs-native-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project
hadoop-client-modules/hadoop-client-minicluster . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
25s{color} | {color:red} hadoop-aws in the patch failed. {color} |
|
[jira] [Commented] (HADOOP-15229) Add FileSystem builder-based openFile() API to match createFile() + S3 Select
[ https://issues.apache.org/jira/browse/HADOOP-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752922#comment-16752922 ] Yuzhou Sun commented on HADOOP-15229: - Sorry I just noticed that: Should SelectInputStream#mark do nothing instead of throwing unsupported? Other than that LGTM +1. Thanks for working on this! > Add FileSystem builder-based openFile() API to match createFile() + S3 Select > - > > Key: HADOOP-15229 > URL: https://issues.apache.org/jira/browse/HADOOP-15229 > Project: Hadoop Common > Issue Type: New Feature > Components: fs, fs/azure, fs/s3 >Affects Versions: 3.2.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Attachments: HADOOP-15229-001.patch, HADOOP-15229-002.patch, > HADOOP-15229-003.patch, HADOOP-15229-004.patch, HADOOP-15229-004.patch, > HADOOP-15229-005.patch, HADOOP-15229-006.patch, HADOOP-15229-007.patch, > HADOOP-15229-009.patch, HADOOP-15229-010.patch, HADOOP-15229-011.patch, > HADOOP-15229-012.patch, HADOOP-15229-013.patch, HADOOP-15229-014.patch, > HADOOP-15229-015.patch, HADOOP-15229-016.patch, HADOOP-15229-017.patch, > HADOOP-15229-018.patch, HADOOP-15229-019.patch > > > Replicate HDFS-1170 and HADOOP-14365 with an API to open files. > A key requirement of this is not HDFS, it's to put in the fadvise policy for > working with object stores, where getting the decision to do a full GET and > TCP abort on seek vs smaller GETs is fundamentally different: the wrong > option can cost you minutes. S3A and Azure both have adaptive policies now > (first backward seek), but they still don't do it that well. > Columnar formats (ORC, Parquet) should be able to say "fs.input.fadvise" > "random" as an option when they open files; I can imagine other options too. > The Builder model of [~eddyxu] is the one to mimic, method for method. > Ideally with as much code reuse as possible -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] BELUGABEHR commented on issue #470: [HADOOP-16073] Use JDK1.7 StandardCharsets
BELUGABEHR commented on issue #470: [HADOOP-16073] Use JDK1.7 StandardCharsets URL: https://github.com/apache/hadoop/pull/470#issuecomment-457793674 I had to make one change... one place was expecting "UTF-8" to be lowercase. Pushed to branch. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16075) Upgrade checkstyle version to 8.16
[
https://issues.apache.org/jira/browse/HADOOP-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752911#comment-16752911
]
Hadoop QA commented on HADOOP-16075:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
61m 38s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 13m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 56s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}167m 26s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
47s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}303m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.TestNameNodeMXBean |
| | hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy |
| | hadoop.hdfs.TestClientMetrics |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-16075 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956399/HADOOP-16075.00.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml |
| uname | Linux 086fcd2657c7 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / dc5af4c |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15847/artifact/out/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15847/testReport/ |
| Max. process+thread count | 2874 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15847/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Upgrade checkstyle version to 8.16
> --
>
> Key: HADOOP-16075
>
[jira] [Commented] (HADOOP-16041) UserAgent string for ABFS
[ https://issues.apache.org/jira/browse/HADOOP-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752888#comment-16752888 ] Sean Mackrory commented on HADOOP-16041: Looks pretty good to me. Test failure is unrelated and probably just flaky given it's fuzzy logic to begin with. We probably want it to be "ABFS/" + VersionInfo..., though, as [~tmarquardt] suggested. I'm fairly certain that the term ABFS uniquely refers to the Hadoop driver, right? So that would be a nice clean way to distinguish Hadoop / ABFS from other ADLS Gen2 clients since the specific formatting of the Hadoop version can vary. I'd also be fine with "Azure Blob FS/" so that we simply replace the 1.0 with the actual Hadoop version (unless 1.0 referred to an API version or something?) > UserAgent string for ABFS > - > > Key: HADOOP-16041 > URL: https://issues.apache.org/jira/browse/HADOOP-16041 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Shweta >Assignee: Shweta >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16041.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14178) Move Mockito up to version 2.x
[ https://issues.apache.org/jira/browse/HADOOP-14178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-14178: -- Attachment: HADOOP-14178.030.patch > Move Mockito up to version 2.x > -- > > Key: HADOOP-14178 > URL: https://issues.apache.org/jira/browse/HADOOP-14178 > Project: Hadoop Common > Issue Type: Sub-task > Components: test >Affects Versions: 2.9.0 >Reporter: Steve Loughran >Assignee: Akira Ajisaka >Priority: Major > Attachments: HADOOP-14178.001.patch, HADOOP-14178.002.patch, > HADOOP-14178.003.patch, HADOOP-14178.004.patch, HADOOP-14178.005-wip.patch, > HADOOP-14178.005-wip2.patch, HADOOP-14178.005-wip3.patch, > HADOOP-14178.005-wip4.patch, HADOOP-14178.005-wip5.patch, > HADOOP-14178.005-wip6.patch, HADOOP-14178.005.patch, HADOOP-14178.006.patch, > HADOOP-14178.007.patch, HADOOP-14178.008.patch, HADOOP-14178.009.patch, > HADOOP-14178.010.patch, HADOOP-14178.011.patch, HADOOP-14178.012.patch, > HADOOP-14178.013.patch, HADOOP-14178.014.patch, HADOOP-14178.015.patch, > HADOOP-14178.016.patch, HADOOP-14178.017.patch, HADOOP-14178.018.patch, > HADOOP-14178.019.patch, HADOOP-14178.020.patch, HADOOP-14178.021.patch, > HADOOP-14178.022.patch, HADOOP-14178.023.patch, HADOOP-14178.024.patch, > HADOOP-14178.025.patch, HADOOP-14178.026.patch, HADOOP-14178.027.patch, > HADOOP-14178.028.patch, HADOOP-14178.029.patch, HADOOP-14178.030.patch > > > I don't know when Hadoop picked up Mockito, but it has been frozen at 1.8.5 > since the switch to maven in 2011. > Mockito is now at version 2.1, [with lots of Java 8 > support|https://github.com/mockito/mockito/wiki/What%27s-new-in-Mockito-2]. > That' s not just defining actions as closures, but in supporting Optional > types, mocking methods in interfaces, etc. > It's only used for testing, and, *provided there aren't regressions*, cost of > upgrade is low. The good news: test tools usually come with good test > coverage. The bad: mockito does go deep into java bytecodes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14178) Move Mockito up to version 2.x
[
https://issues.apache.org/jira/browse/HADOOP-14178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752878#comment-16752878
]
Masatake Iwasaki commented on HADOOP-14178:
---
[~ajisakaa] I'm looking into the patch. 029 has simple confilicts in 5 files
with current trunk. I'm attaching rebased patch as 030 for your convenience.
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/checker/TestThrottledAsyncCheckerTimeout.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/test/java/org/apache/hadoop/yarn/service/MockRunningServiceContext.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/test/java/org/apache/hadoop/yarn/service/ServiceTestUtils.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.ja
{code}
---
a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java
+++
b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java
..(snip)
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
..(snip)
- Mockito.any(SocketAddress.class), Mockito.anyInt());
+ any(SocketAddress.class), Mockito.anyInt());
{code}
Added ArgumentMatchers.anyInt is not used.
Compile fails on recently added
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/mapreduce/MockJob.java .
> Move Mockito up to version 2.x
> --
>
> Key: HADOOP-14178
> URL: https://issues.apache.org/jira/browse/HADOOP-14178
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: test
>Affects Versions: 2.9.0
>Reporter: Steve Loughran
>Assignee: Akira Ajisaka
>Priority: Major
> Attachments: HADOOP-14178.001.patch, HADOOP-14178.002.patch,
> HADOOP-14178.003.patch, HADOOP-14178.004.patch, HADOOP-14178.005-wip.patch,
> HADOOP-14178.005-wip2.patch, HADOOP-14178.005-wip3.patch,
> HADOOP-14178.005-wip4.patch, HADOOP-14178.005-wip5.patch,
> HADOOP-14178.005-wip6.patch, HADOOP-14178.005.patch, HADOOP-14178.006.patch,
> HADOOP-14178.007.patch, HADOOP-14178.008.patch, HADOOP-14178.009.patch,
> HADOOP-14178.010.patch, HADOOP-14178.011.patch, HADOOP-14178.012.patch,
> HADOOP-14178.013.patch, HADOOP-14178.014.patch, HADOOP-14178.015.patch,
> HADOOP-14178.016.patch, HADOOP-14178.017.patch, HADOOP-14178.018.patch,
> HADOOP-14178.019.patch, HADOOP-14178.020.patch, HADOOP-14178.021.patch,
> HADOOP-14178.022.patch, HADOOP-14178.023.patch, HADOOP-14178.024.patch,
> HADOOP-14178.025.patch, HADOOP-14178.026.patch, HADOOP-14178.027.patch,
> HADOOP-14178.028.patch, HADOOP-14178.029.patch, HADOOP-14178.030.patch
>
>
> I don't know when Hadoop picked up Mockito, but it has been frozen at 1.8.5
> since the switch to maven in 2011.
> Mockito is now at version 2.1, [with lots of Java 8
> support|https://github.com/mockito/mockito/wiki/What%27s-new-in-Mockito-2].
> That' s not just defining actions as closures, but in supporting Optional
> types, mocking methods in interfaces, etc.
> It's only used for testing, and, *provided there aren't regressions*, cost of
> upgrade is low. The good news: test tools usually come with good test
> coverage. The bad: mockito does go deep into java bytecodes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16041) UserAgent string for ABFS
[
https://issues.apache.org/jira/browse/HADOOP-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752877#comment-16752877
]
Hadoop QA commented on HADOOP-16041:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 15s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 51s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 12s{color}
| {color:red} hadoop-azure in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.azure.TestClientThrottlingAnalyzer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-16041 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956409/HADOOP-16041.001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 5193fafcc536 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / fb69519 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15848/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15848/testReport/ |
| Max. process+thread count | 305 (vs. ulimit of 1) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15848/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
[GitHub] elek commented on issue #470: [HADOOP-16073] Use JDK1.7 StandardCharsets
elek commented on issue #470: [HADOOP-16073] Use JDK1.7 StandardCharsets URL: https://github.com/apache/hadoop/pull/470#issuecomment-457784813 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752870#comment-16752870 ] BELUGA BEHR commented on HADOOP-16073: -- https://github.com/apache/hadoop/pull/470 > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Priority: Major > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] BELUGABEHR opened a new pull request #470: [HADOOP-16073] Use JDK1.7 StandardCharsets
BELUGABEHR opened a new pull request #470: [HADOOP-16073] Use JDK1.7 StandardCharsets URL: https://github.com/apache/hadoop/pull/470 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-16073: - Summary: Use JDK1.7 StandardCharsets (was: Use JDK1.7 StandardCharset) > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Priority: Major > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16041) UserAgent string for ABFS
[ https://issues.apache.org/jira/browse/HADOOP-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752827#comment-16752827 ] Shweta commented on HADOOP-16041: - Thanks for the discussions [~ste...@apache.org], [~mackrorysd], [~tmarquardt] and [~DanielZhou]. Patch001 contains the change to reflect the VersionInfo in User Agent String. Please review. Thanks. > UserAgent string for ABFS > - > > Key: HADOOP-16041 > URL: https://issues.apache.org/jira/browse/HADOOP-16041 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Shweta >Assignee: Shweta >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16041.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16041) UserAgent string for ABFS
[ https://issues.apache.org/jira/browse/HADOOP-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shweta updated HADOOP-16041: Attachment: HADOOP-16041.001.patch Status: Patch Available (was: Open) > UserAgent string for ABFS > - > > Key: HADOOP-16041 > URL: https://issues.apache.org/jira/browse/HADOOP-16041 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Shweta >Assignee: Shweta >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16041.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11223) Offer a read-only conf alternative to new Configuration()
[
https://issues.apache.org/jira/browse/HADOOP-11223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752809#comment-16752809
]
Michael Miller commented on HADOOP-11223:
-
I wrote a class that extends Configuration called ImmutableConfiguration and a
Test. Please let me know if this can be used anywhere or what branch I can make
a pull request against.
I have them in my local github
[here|https://github.com/milleruntime/accumulo/blob/immutableConf/core/src/main/java/org/apache/accumulo/core/conf/ImmutableConfiguration.java].
> Offer a read-only conf alternative to new Configuration()
> -
>
> Key: HADOOP-11223
> URL: https://issues.apache.org/jira/browse/HADOOP-11223
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Gopal V
>Assignee: Varun Saxena
>Priority: Major
> Labels: Performance
> Attachments: HADOOP-11223.001.patch
>
>
> new Configuration() is called from several static blocks across Hadoop.
> This is incredibly inefficient, since each one of those involves primarily
> XML parsing at a point where the JIT won't be triggered & interpreter mode is
> essentially forced on the JVM.
> The alternate solution would be to offer a {{Configuration::getDefault()}}
> alternative which disallows any modifications.
> At the very least, such a method would need to be called from
> # org.apache.hadoop.io.nativeio.NativeIO::()
> # org.apache.hadoop.security.SecurityUtil::()
> # org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider::
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16074) WASB: Update container not found error code
[ https://issues.apache.org/jira/browse/HADOOP-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-16074: - Attachment: HADOOP-16074-001.patch > WASB: Update container not found error code > --- > > Key: HADOOP-16074 > URL: https://issues.apache.org/jira/browse/HADOOP-16074 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Da Zhou >Assignee: Da Zhou >Priority: Major > Attachments: HADOOP-16074-001.patch > > > Due to service error code change, container is not automatically created for > WRITE operation in WASB. > Fix: need to update to correct error code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16074) WASB: Update container not found error code
[ https://issues.apache.org/jira/browse/HADOOP-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752754#comment-16752754 ] Da Zhou commented on HADOOP-16074: -- Submitting 001 patch: * Replaced error code check with the correct one. * Updated tests to cover the test case that WRITE type operation on non-existing container. Wasb tests passed: Tests run: 245, Failures: 0, Errors: 0, Skipped: 11 Tests run: 620, Failures: 0, Errors: 0, Skipped: 66: Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 > WASB: Update container not found error code > --- > > Key: HADOOP-16074 > URL: https://issues.apache.org/jira/browse/HADOOP-16074 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Da Zhou >Assignee: Da Zhou >Priority: Major > Attachments: HADOOP-16074-001.patch > > > Due to service error code change, container is not automatically created for > WRITE operation in WASB. > Fix: need to update to correct error code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16076) SPNEGO+SSL Client Connections with HttpClient Broken
Larry McCay created HADOOP-16076: Summary: SPNEGO+SSL Client Connections with HttpClient Broken Key: HADOOP-16076 URL: https://issues.apache.org/jira/browse/HADOOP-16076 Project: Hadoop Common Issue Type: Bug Components: build, security Affects Versions: 3.2.0 Reporter: Larry McCay Assignee: Larry McCay Client connections with HttpClient to a SPNEGO secured endpoint with TLS enabled break due to a misrepresentation of the SPN to include HTTPS instead of just HTTP. The current use of HTTPClient 4.5.2 is affected by HTTPCLIENT-1712 and breaks SPNEGO with HTTPS endpoints since it include the httpS in the principal name. We need to migrate to at least 4.5.3 as we have tested with that version and observed it fixing the issue. Need to do some due diligence to determine the cleanest version to upgrade to but will provide a patch in a day or so. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16075) Upgrade checkstyle version to 8.16
[ https://issues.apache.org/jira/browse/HADOOP-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752695#comment-16752695 ] Bharat Viswanadham commented on HADOOP-16075: - Any reason why need this? Just asking if it caused any issues with the current version. > Upgrade checkstyle version to 8.16 > -- > > Key: HADOOP-16075 > URL: https://issues.apache.org/jira/browse/HADOOP-16075 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.2.0 >Reporter: Dinesh Chitlangia >Assignee: Dinesh Chitlangia >Priority: Minor > Attachments: HADOOP-16075.00.patch > > > Jira aims to upgrade checkstyle version from 8.8 to 8.16. > It is a minor upgrade with some bug fixes in checkstyle and its a negligible > risk change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16075) Upgrade checkstyle version to 8.16
[ https://issues.apache.org/jira/browse/HADOOP-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752702#comment-16752702 ] Dinesh Chitlangia commented on HADOOP-16075: [~bharatviswa] No issues were caused. Upgrading minor releases is easier than going for a major change at once :) > Upgrade checkstyle version to 8.16 > -- > > Key: HADOOP-16075 > URL: https://issues.apache.org/jira/browse/HADOOP-16075 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.2.0 >Reporter: Dinesh Chitlangia >Assignee: Dinesh Chitlangia >Priority: Minor > Attachments: HADOOP-16075.00.patch > > > Jira aims to upgrade checkstyle version from 8.8 to 8.16. > It is a minor upgrade with some bug fixes in checkstyle and its a negligible > risk change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16075) Upgrade checkstyle version to 8.16
[
https://issues.apache.org/jira/browse/HADOOP-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752686#comment-16752686
]
Dinesh Chitlangia commented on HADOOP-16075:
Verified the patch manually by
{code:java}
mvn checkstyle:checkstyle
{code}
> Upgrade checkstyle version to 8.16
> --
>
> Key: HADOOP-16075
> URL: https://issues.apache.org/jira/browse/HADOOP-16075
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Dinesh Chitlangia
>Assignee: Dinesh Chitlangia
>Priority: Minor
> Attachments: HADOOP-16075.00.patch
>
>
> Jira aims to upgrade checkstyle version from 8.8 to 8.16.
> It is a minor upgrade with some bug fixes in checkstyle and its a negligible
> risk change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16075) Upgrade checkstyle version to 8.16
Dinesh Chitlangia created HADOOP-16075: -- Summary: Upgrade checkstyle version to 8.16 Key: HADOOP-16075 URL: https://issues.apache.org/jira/browse/HADOOP-16075 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: Dinesh Chitlangia Assignee: Dinesh Chitlangia Jira aims to upgrade checkstyle version from 8.8 to 8.16. It is a minor upgrade with some bug fixes in checkstyle and its a negligible risk change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16075) Upgrade checkstyle version to 8.16
[ https://issues.apache.org/jira/browse/HADOOP-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Chitlangia updated HADOOP-16075: --- Affects Version/s: 3.2.0 > Upgrade checkstyle version to 8.16 > -- > > Key: HADOOP-16075 > URL: https://issues.apache.org/jira/browse/HADOOP-16075 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.2.0 >Reporter: Dinesh Chitlangia >Assignee: Dinesh Chitlangia >Priority: Minor > Attachments: HADOOP-16075.00.patch > > > Jira aims to upgrade checkstyle version from 8.8 to 8.16. > It is a minor upgrade with some bug fixes in checkstyle and its a negligible > risk change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16075) Upgrade checkstyle version to 8.16
[ https://issues.apache.org/jira/browse/HADOOP-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Chitlangia updated HADOOP-16075: --- Attachment: HADOOP-16075.00.patch Status: Patch Available (was: Open) > Upgrade checkstyle version to 8.16 > -- > > Key: HADOOP-16075 > URL: https://issues.apache.org/jira/browse/HADOOP-16075 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Dinesh Chitlangia >Assignee: Dinesh Chitlangia >Priority: Minor > Attachments: HADOOP-16075.00.patch > > > Jira aims to upgrade checkstyle version from 8.8 to 8.16. > It is a minor upgrade with some bug fixes in checkstyle and its a negligible > risk change. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16074) WASB: Update container not found error code
[ https://issues.apache.org/jira/browse/HADOOP-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-16074: - Description: Due to service error code change, container is not automatically created for WRITE operation in WASB. Fix: need to update correct error code. was: Due to service error code change, container is not automatically created for WRITE operation in WASB. Fix: need to update the dependency of Azure Storage SDK to retrieve the correct error code. > WASB: Update container not found error code > --- > > Key: HADOOP-16074 > URL: https://issues.apache.org/jira/browse/HADOOP-16074 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Da Zhou >Assignee: Da Zhou >Priority: Major > > Due to service error code change, container is not automatically created for > WRITE operation in WASB. > Fix: need to update correct error code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16074) WASB: Update container not found error code
[ https://issues.apache.org/jira/browse/HADOOP-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-16074: - Description: Due to service error code change, container is not automatically created for WRITE operation in WASB. Fix: need to update to correct error code. was: Due to service error code change, container is not automatically created for WRITE operation in WASB. Fix: need to update correct error code. > WASB: Update container not found error code > --- > > Key: HADOOP-16074 > URL: https://issues.apache.org/jira/browse/HADOOP-16074 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Da Zhou >Assignee: Da Zhou >Priority: Major > > Due to service error code change, container is not automatically created for > WRITE operation in WASB. > Fix: need to update to correct error code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15711) Fix branch-2 builds
[
https://issues.apache.org/jira/browse/HADOOP-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752624#comment-16752624
]
Allen Wittenauer commented on HADOOP-15711:
---
Granted, I haven't used Hadoop in months and months, but ...
That's probably a discussion for common. It's a pseudo-incompatible change.
It carries quite a few risks. branch-2 is supposed to be JDK7 compatible and
it might be possible for JDK8 code to slip in. Committing those changes will
also make the binary releases built on JDK8 and require newer versions of Linux
shared libraries. On the flip side, Trusty goes EOL in a few months and there
should be some work on moving to something newer on the OS side anyway.
Frankly, it might be an easier discussion to finally just EOL all of branch-2,
given some of the other problems. [~ajisakaa] can shed more light on those
issues.
> Fix branch-2 builds
> ---
>
> Key: HADOOP-15711
> URL: https://issues.apache.org/jira/browse/HADOOP-15711
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Jonathan Hung
>Priority: Critical
> Attachments: HADOOP-15711.001.branch-2.patch
>
>
> Branch-2 builds have been disabled for a while:
> https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/
> A test run here causes hdfs tests to hang:
> https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86-jhung/4/
> Running hadoop-hdfs tests locally reveal some errors such
> as:{noformat}[ERROR]
> testComplexAppend2(org.apache.hadoop.hdfs.TestFileAppend2) Time elapsed:
> 0.059 s <<< ERROR!
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:714)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1128)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:174)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1172)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:403)
> at
> org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:234)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:1080)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:883)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:514)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:473)
> at
> org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend(TestFileAppend2.java:489)
> at
> org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend2(TestFileAppend2.java:543)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){noformat}
> I was able to get more tests passing locally by increasing the max user
> process count on my machine. But the error suggests that there's an issue in
> the tests themselves. Not sure if the error seen locally is the same reason
> as why jenkins builds are failing, I wasn't able to confirm based on the
> jenkins builds' lack of output.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15711) Fix branch-2 builds
[
https://issues.apache.org/jira/browse/HADOOP-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752594#comment-16752594
]
Jonathan Hung commented on HADOOP-15711:
My proposal is to port HADOOP-14816 and -HADOOP-15610- to branch-2 to use
openjdk8 in branch-2 instead of openjdk7. Any objections?
> Fix branch-2 builds
> ---
>
> Key: HADOOP-15711
> URL: https://issues.apache.org/jira/browse/HADOOP-15711
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Jonathan Hung
>Priority: Critical
> Attachments: HADOOP-15711.001.branch-2.patch
>
>
> Branch-2 builds have been disabled for a while:
> https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/
> A test run here causes hdfs tests to hang:
> https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86-jhung/4/
> Running hadoop-hdfs tests locally reveal some errors such
> as:{noformat}[ERROR]
> testComplexAppend2(org.apache.hadoop.hdfs.TestFileAppend2) Time elapsed:
> 0.059 s <<< ERROR!
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:714)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1128)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:174)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1172)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:403)
> at
> org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:234)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:1080)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:883)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:514)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:473)
> at
> org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend(TestFileAppend2.java:489)
> at
> org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend2(TestFileAppend2.java:543)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){noformat}
> I was able to get more tests passing locally by increasing the max user
> process count on my machine. But the error suggests that there's an issue in
> the tests themselves. Not sure if the error seen locally is the same reason
> as why jenkins builds are failing, I wasn't able to confirm based on the
> jenkins builds' lack of output.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16074) WASB: Update container not found error code
[ https://issues.apache.org/jira/browse/HADOOP-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-16074: - Description: Due to service error code change, container is not automatically created for WRITE operation in WASB. Fix: need to update the dependency of Azure Storage SDK to retrieve the correct error code. was: Due to service error code change, container is not automatically created for WRITE operation in WASB. Fix: need to update the corresponding error code. > WASB: Update container not found error code > --- > > Key: HADOOP-16074 > URL: https://issues.apache.org/jira/browse/HADOOP-16074 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Da Zhou >Assignee: Da Zhou >Priority: Major > > Due to service error code change, container is not automatically created for > WRITE operation in WASB. > Fix: need to update the dependency of Azure Storage SDK to retrieve the > correct error code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16064) Load configuration values from external sources
[
https://issues.apache.org/jira/browse/HADOOP-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752582#comment-16752582
]
Larry McCay commented on HADOOP-16064:
--
Thanks for pointing me to this [~ste...@apache.org].
I do get a bit concerned about different sources of configuration.
Keep in mind that config files are generally locked down for root with proper
file permissions.
We can't allow overrides to come from unsecured sources.
I'll need to give this some more thought and the patch a review to have any
more meaningful comments. :)
> Load configuration values from external sources
> ---
>
> Key: HADOOP-16064
> URL: https://issues.apache.org/jira/browse/HADOOP-16064
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HADOOP-16064.001.patch
>
>
> This is a proposal to improve the Configuration.java to load configuration
> from external sources (kubernetes config map, external http reqeust, any
> cluster manager like ambari, etc.)
> I will attach a patch to illustrate the proposed solution, but please comment
> the concept first, the patch is just poc and not fully implemented.
> *Goals:*
> * Load the configuration files (core-site.xml/hdfs-site.xml/...) from
> external locations instead of the classpath (classpath remains the default)
> * Make the configuration loading extensible
> * Make it in an backward-compatible way with minimal change in the existing
> Configuration.java
> *Use-cases:*
> 1.) load configuration from the namenode ([http://namenode:9878/conf]). With
> this approach only the namenode should be configured, other components
> require only the url of the namenode
> 2.) Read configuration directly from kubernetes config-map (or mesos)
> 3.) Read configuration from any external cluster management (such as Apache
> Ambari or any equivalent)
> 4.) as of now in the hadoop docker images we transform environment variables
> (such as HDFS-SITE.XML_fs.defaultFs) to configuration xml files with the help
> of a python script. With the proposed implementation it would be possible to
> read the configuration directly from the system environment variables.
> *Problem:*
> The existing Configuration.java can read configuration from multiple sources.
> But most of the time it's used to load predefined config names
> ("core-site.xml" and "hdfs-site.xml") without configuration location. In this
> case the files will be loaded from the classpath.
> I propose to add additional option to define the default location of
> core-site.xml and hdfs-site.xml (any configuration which is defined by string
> name) to use external sources in the classpath.
> The configuration loading requires implementation + configuration (where are
> the external configs). We can't use regular configuration to configure the
> config loader (chicken/egg).
> I propose to use a new environment variable HADOOP_CONF_SOURCE
> The environment variable could contain a URL, where the schema of the url can
> define the config source and all the other parts can configure the access to
> the resource.
> Examples:
> HADOOP_CONF_SOURCE=hadoop-[http://namenode:9878/conf]
> HADOOP_CONF_SOURCE=env://prefix
> HADOOP_CONF_SOURCE=k8s://config-map-name
> The ConfigurationSource interface can be as easy as:
> {code:java}
> /**
> * Interface to load hadoop configuration from custom location.
> */
> public interface ConfigurationSource {
> /**
> * Method will be called one with the defined configuration url.
> *
> * @param uri
> */
> void initialize(URI uri) throws IOException;
> /**
> * Method will be called to load a specific configuration resource.
> *
> * @param name of the configuration resource (eg. hdfs-site.xml)
> * @return List of loaded configuraiton key and values.
> */
> List readConfiguration(String name);
> }{code}
> We can choose the right implementation based the schema of the uri and with
> Java Service Provider Interface mechanism
> (META-INF/services/org.apache.hadoop.conf.ConfigurationSource)
> It could be with minimal modification in the Configuration.java (see the
> attached patch as an example)
> The patch contains two example implementation:
> *hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/location/Env.java*
> This can load configuration from environment variables based on a naming
> convention (eg. HDFS-SITE.XML_hdfs.dfs.key=value)
> *hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/location/HadoopWeb.java*
> This implementation can load the configuration from a /conf servlet of any
> Hadoop components.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HADOOP-16074) WASB: Update container not found error code
Da Zhou created HADOOP-16074: Summary: WASB: Update container not found error code Key: HADOOP-16074 URL: https://issues.apache.org/jira/browse/HADOOP-16074 Project: Hadoop Common Issue Type: Bug Components: fs/azure Affects Versions: 3.2.0 Reporter: Da Zhou Assignee: Da Zhou Due to service error code change, container is not automatically created for WRITE operation in WASB. Fix: need to update the corresponding error code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15998) Jar validation bash scripts don't work on Windows due to platform differences (colons in paths, \r\n)
[ https://issues.apache.org/jira/browse/HADOOP-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752520#comment-16752520 ] Giovanni Matteo Fumarola commented on HADOOP-15998: --- Thanks [~briangru]. [^HADOOP-15998.v4.patch]solves the issue that [~busbey] mentioned early. The integration tests [https://builds.apache.org/job/PreCommit-HADOOP-Build/15842/artifact/out/patch-shadedclient.txt] does not show any more errors. This is the first time I saw spellcheck errors from Yetus. [www.shellcheck.net|https://www.shellcheck.net/] provides a live to find and fix those errors. NIT: Please keep the old patch in the Jira for history purposes. > Jar validation bash scripts don't work on Windows due to platform differences > (colons in paths, \r\n) > - > > Key: HADOOP-15998 > URL: https://issues.apache.org/jira/browse/HADOOP-15998 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.2.0, 3.3.0 > Environment: Windows 10 > Visual Studio 2017 >Reporter: Brian Grunkemeyer >Assignee: Brian Grunkemeyer >Priority: Blocker > Labels: build, windows > Fix For: 3.3.0 > > Attachments: HADOOP-15998.v4.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Building Hadoop fails on Windows due to a few shell scripts that make invalid > assumptions: > 1) Colon shouldn't be used to separate multiple paths in command line > parameters. Colons occur in Windows paths. > 2) Shell scripts that rely on running external tools need to deal with > carriage return - line feed differences (lines ending in \r\n, not just \n) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16073) Use JDK1.7 StandardCharset
BELUGA BEHR created HADOOP-16073: Summary: Use JDK1.7 StandardCharset Key: HADOOP-16073 URL: https://issues.apache.org/jira/browse/HADOOP-16073 Project: Hadoop Common Issue Type: Improvement Components: streaming, tools Affects Versions: 3.2.0 Reporter: BELUGA BEHR Use Java 1.7 [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11223) Offer a read-only conf alternative to new Configuration()
[
https://issues.apache.org/jira/browse/HADOOP-11223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752437#comment-16752437
]
Keith Turner commented on HADOOP-11223:
---
It would be nice if Configuraiton had a static method like
Collections.unmodifiableXXX.
{code:java}
Configuraiton conf = new Configuration();
//wrap conf with a class that throws exception on set
Configuraiton readonlyConf = Configuration.umodifiable(conf);
{code}
> Offer a read-only conf alternative to new Configuration()
> -
>
> Key: HADOOP-11223
> URL: https://issues.apache.org/jira/browse/HADOOP-11223
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Gopal V
>Assignee: Varun Saxena
>Priority: Major
> Labels: Performance
> Attachments: HADOOP-11223.001.patch
>
>
> new Configuration() is called from several static blocks across Hadoop.
> This is incredibly inefficient, since each one of those involves primarily
> XML parsing at a point where the JIT won't be triggered & interpreter mode is
> essentially forced on the JVM.
> The alternate solution would be to offer a {{Configuration::getDefault()}}
> alternative which disallows any modifications.
> At the very least, such a method would need to be called from
> # org.apache.hadoop.io.nativeio.NativeIO::()
> # org.apache.hadoop.security.SecurityUtil::()
> # org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider::
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-11223) Offer a read-only conf alternative to new Configuration()
[
https://issues.apache.org/jira/browse/HADOOP-11223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752437#comment-16752437
]
Keith Turner edited comment on HADOOP-11223 at 1/25/19 4:57 PM:
It would be nice if Configuraiton had a static method like
Collections.unmodifiableXXX.
{code:java}
Configuraiton conf = new Configuration();
//wrap conf with a class that throws exception on set
Configuration readonlyConf = Configuration.umodifiable(conf);
{code}
was (Author: kturner):
It would be nice if Configuraiton had a static method like
Collections.unmodifiableXXX.
{code:java}
Configuraiton conf = new Configuration();
//wrap conf with a class that throws exception on set
Configuraiton readonlyConf = Configuration.umodifiable(conf);
{code}
> Offer a read-only conf alternative to new Configuration()
> -
>
> Key: HADOOP-11223
> URL: https://issues.apache.org/jira/browse/HADOOP-11223
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Gopal V
>Assignee: Varun Saxena
>Priority: Major
> Labels: Performance
> Attachments: HADOOP-11223.001.patch
>
>
> new Configuration() is called from several static blocks across Hadoop.
> This is incredibly inefficient, since each one of those involves primarily
> XML parsing at a point where the JIT won't be triggered & interpreter mode is
> essentially forced on the JVM.
> The alternate solution would be to offer a {{Configuration::getDefault()}}
> alternative which disallows any modifications.
> At the very least, such a method would need to be called from
> # org.apache.hadoop.io.nativeio.NativeIO::()
> # org.apache.hadoop.security.SecurityUtil::()
> # org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider::
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15281) Distcp to add no-rename copy option
[
https://issues.apache.org/jira/browse/HADOOP-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752319#comment-16752319
]
Andrew Olson commented on HADOOP-15281:
---
[~ste...@apache.org] Got a +1 from Jenkins on this. Let me know if any code
changes need to be made.
> Distcp to add no-rename copy option
> ---
>
> Key: HADOOP-15281
> URL: https://issues.apache.org/jira/browse/HADOOP-15281
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
>Assignee: Andrew Olson
>Priority: Major
> Attachments: HADOOP-15281-001.patch, HADOOP-15281-002.patch,
> HADOOP-15281-003.patch
>
>
> Currently Distcp uploads a file by two strategies
> # append parts
> # copy to temp then rename
> option 2 executes the following sequence in {{promoteTmpToTarget}}
> {code}
> if ((fs.exists(target) && !fs.delete(target, false))
> || (!fs.exists(target.getParent()) && !fs.mkdirs(target.getParent()))
> || !fs.rename(tmpTarget, target)) {
> throw new IOException("Failed to promote tmp-file:" + tmpTarget
> + " to: " + target);
> }
> {code}
> For any object store, that's a lot of HTTP requests; for S3A you are looking
> at 12+ requests and an O(data) copy call.
> This is not a good upload strategy for any store which manifests its output
> atomically at the end of the write().
> Proposed: add a switch to write direct to the dest path. either a conf option
> or a CLI option
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16055) Upgrade AWS SDK to 1.11.271 in branch-2
[ https://issues.apache.org/jira/browse/HADOOP-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752212#comment-16752212 ] Akira Ajisaka commented on HADOOP-16055: Committed this to branch-2. Thanks [~ste...@apache.org]! The patch for branch-2 applies to branch-2.9 cleanly. Uploaded the same patch to kick precommit job against branch-2.9. > Upgrade AWS SDK to 1.11.271 in branch-2 > --- > > Key: HADOOP-16055 > URL: https://issues.apache.org/jira/browse/HADOOP-16055 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Blocker > Fix For: 2.10.0 > > Attachments: HADOOP-16055-branch-2-01.patch, > HADOOP-16055-branch-2.8-01.patch, HADOOP-16055-branch-2.8-02.patch, > HADOOP-16055-branch-2.9-01.patch > > > Per HADOOP-13794, we must exclude the JSON license. > The upgrade will contain incompatible changes, however, the license issue is > much more important. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16055) Upgrade AWS SDK to 1.11.271 in branch-2
[ https://issues.apache.org/jira/browse/HADOOP-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-16055: --- Attachment: HADOOP-16055-branch-2.9-01.patch > Upgrade AWS SDK to 1.11.271 in branch-2 > --- > > Key: HADOOP-16055 > URL: https://issues.apache.org/jira/browse/HADOOP-16055 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Blocker > Fix For: 2.10.0 > > Attachments: HADOOP-16055-branch-2-01.patch, > HADOOP-16055-branch-2.8-01.patch, HADOOP-16055-branch-2.8-02.patch, > HADOOP-16055-branch-2.9-01.patch > > > Per HADOOP-13794, we must exclude the JSON license. > The upgrade will contain incompatible changes, however, the license issue is > much more important. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16055) Upgrade AWS SDK to 1.11.271 in branch-2
[ https://issues.apache.org/jira/browse/HADOOP-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-16055: --- Fix Version/s: 2.10.0 > Upgrade AWS SDK to 1.11.271 in branch-2 > --- > > Key: HADOOP-16055 > URL: https://issues.apache.org/jira/browse/HADOOP-16055 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Blocker > Fix For: 2.10.0 > > Attachments: HADOOP-16055-branch-2-01.patch, > HADOOP-16055-branch-2.8-01.patch, HADOOP-16055-branch-2.8-02.patch > > > Per HADOOP-13794, we must exclude the JSON license. > The upgrade will contain incompatible changes, however, the license issue is > much more important. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16055) Upgrade AWS SDK to 1.11.271 in branch-2
[ https://issues.apache.org/jira/browse/HADOOP-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-16055: --- Target Version/s: 2.10.0, 2.7.8, 2.8.6, 2.9.3 (was: 2.7.8, 2.8.6) > Upgrade AWS SDK to 1.11.271 in branch-2 > --- > > Key: HADOOP-16055 > URL: https://issues.apache.org/jira/browse/HADOOP-16055 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Blocker > Fix For: 2.10.0 > > Attachments: HADOOP-16055-branch-2-01.patch, > HADOOP-16055-branch-2.8-01.patch, HADOOP-16055-branch-2.8-02.patch > > > Per HADOOP-13794, we must exclude the JSON license. > The upgrade will contain incompatible changes, however, the license issue is > much more important. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16055) Upgrade AWS SDK to 1.11.271 in branch-2
[ https://issues.apache.org/jira/browse/HADOOP-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-16055: --- Summary: Upgrade AWS SDK to 1.11.271 in branch-2 (was: Upgrade AWS SDK to fix license issue in branch-2.8 and branch-2.7) > Upgrade AWS SDK to 1.11.271 in branch-2 > --- > > Key: HADOOP-16055 > URL: https://issues.apache.org/jira/browse/HADOOP-16055 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Blocker > Attachments: HADOOP-16055-branch-2-01.patch, > HADOOP-16055-branch-2.8-01.patch, HADOOP-16055-branch-2.8-02.patch > > > Per HADOOP-13794, we must exclude the JSON license. > The upgrade will contain incompatible changes, however, the license issue is > much more important. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16055) Upgrade AWS SDK to fix license issue in branch-2.8 and branch-2.7
[
https://issues.apache.org/jira/browse/HADOOP-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752205#comment-16752205
]
Akira Ajisaka commented on HADOOP-16055:
I re-ran the test locally and it passed.
{noformat}
$ mvn verify -Dtest=skip -Dit.test=ITestS3AContractRootDir
$ mvn verify -Dtest=skip -Dit.test=ITestS3AContractRootDir -Ddynamo -Ds3guard
{noformat}
Committing this to branch-2.
> Upgrade AWS SDK to fix license issue in branch-2.8 and branch-2.7
> -
>
> Key: HADOOP-16055
> URL: https://issues.apache.org/jira/browse/HADOOP-16055
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Blocker
> Attachments: HADOOP-16055-branch-2-01.patch,
> HADOOP-16055-branch-2.8-01.patch, HADOOP-16055-branch-2.8-02.patch
>
>
> Per HADOOP-13794, we must exclude the JSON license.
> The upgrade will contain incompatible changes, however, the license issue is
> much more important.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16058) S3A tests to include Terasort
[
https://issues.apache.org/jira/browse/HADOOP-16058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752155#comment-16752155
]
Steve Loughran commented on HADOOP-16058:
-
javac warning is just using Path.makeQualified; test failure is
{code}
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 02:00 h
[INFO] Finished at: 2019-01-21T19:24:59+00:00
[INFO] Final Memory: 25M/488M
[INFO]
[WARNING] The requested profile "parallel-tests" could not be activated because
it does not exist.
[WARNING] The requested profile "native" could not be activated because it does
not exist.
[WARNING] The requested profile "yarn-ui" could not be activated because it
does not exist.
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test) on
project hadoop-mapreduce-client-jobclient: There was a timeout or other error
in the fork -> [Help 1]
{code}
Either this is a regression, or there's a test which is turning out to be
brittle
> S3A tests to include Terasort
> -
>
> Key: HADOOP-16058
> URL: https://issues.apache.org/jira/browse/HADOOP-16058
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Attachments: HADOOP-16058-001.patch, HADOOP-16058-002.patch
>
>
> Add S3A tests to run terasort for the magic and directory committers.
> MAPREDUCE-7091 is a requirement for this
> Bonus feature: print the results to see which committers are faster in the
> specific test setup. As that's a function of latency to the store, bandwidth
> and size of jobs, it's not at all meaningful, just interesting.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
