[jira] [Commented] (HADOOP-10783) apache-commons-lang.jar 2.6 does not support FreeBSD -upgrade to 3.x needed
[ https://issues.apache.org/jira/browse/HADOOP-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481429#comment-16481429 ] Takanobu Asanuma commented on HADOOP-10783: --- The build failed due to out of heap memory again... Would somebody please advise me about it? > apache-commons-lang.jar 2.6 does not support FreeBSD -upgrade to 3.x needed > --- > > Key: HADOOP-10783 > URL: https://issues.apache.org/jira/browse/HADOOP-10783 > Project: Hadoop Common > Issue Type: Bug >Reporter: Dmitry Sivachenko >Assignee: Takanobu Asanuma >Priority: Major > Attachments: HADOOP-10783.2.patch, HADOOP-10783.3.patch, > HADOOP-10783.4.patch, HADOOP-10783.5.patch, commons-lang3_1.patch > > > Hadoop-2.4.1 ships with apache-commons.jar version 2.6. > It does not support FreeBSD (IS_OS_UNIX returns False). > This is fixed in recent versions of apache-commons.jar > Please update apache-commons.jar to recent version so it correctly recognizes > FreeBSD as UNIX-like system. > Right now I get in datanode's log: > 2014-07-04 11:58:10,459 DEBUG > org.apache.hadoop.hdfs.server.datanode.ShortCircui > tRegistry: Disabling ShortCircuitRegistry > java.io.IOException: The OS is not UNIX. > at > org.apache.hadoop.io.nativeio.SharedFileDescriptorFactory.create(SharedFileDescriptorFactory.java:77) > at > org.apache.hadoop.hdfs.server.datanode.ShortCircuitRegistry.(ShortCircuitRegistry.java:169) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.initDataXceiver(DataNode.java:583) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:771) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:289) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1931) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1818) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1865) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2041) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2065) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15458) org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows
[
https://issues.apache.org/jira/browse/HADOOP-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481388#comment-16481388
]
genericqa commented on HADOOP-15458:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 20m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
24s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
45s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
11s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:f667ef1 |
| JIRA Issue | HADOOP-15458 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924216/HADOOP-15458-branch-2.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 0ec075b05c61 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2 / 8f45d11 |
| maven | version: Apache Maven 3.3.9
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) |
| Default Java | 1.7.0_171 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14661/testReport/ |
| Max. process+thread count | 1664 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14661/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on
> Windows
> ---
>
> Key: HADOOP-15458
> URL: https://issues.apache.org/jira/browse/HADOOP-15458
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Xiao Liang
>Assignee: Xiao Liang
>Priority: Major
> Labels: windows
> Attachments: HADOOP-15458-branch-2.000.patch,
> HADOOP-15458-branch-2.001.patch,
[jira] [Commented] (HADOOP-15458) org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows
[
https://issues.apache.org/jira/browse/HADOOP-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481345#comment-16481345
]
Xiao Liang commented on HADOOP-15458:
-
Thanks [~elgoiri] and [~ste...@apache.org] for helping review the patch.
[^HADOOP-15458.001.patch] is for trunk and [^HADOOP-15458-branch-2.001.patch]
is for branch-2, and I ran the tests on Windows,
TestLocalFileSystem#testFSOutputStreamBuilder passed (Administrator permission
is needed for branch-2, while trunk can pass with normal user permission, don't
know the root cause yet):
{color:#14892c}[INFO]
---{color}
{color:#14892c}[INFO] T E S T S{color}
{color:#14892c}[INFO]
---{color}
{color:#14892c}[INFO] Running org.apache.hadoop.fs.TestLocalFileSystem{color}
{color:#14892c}[WARNING] Tests run: 21, Failures: 0, Errors: 0, Skipped: 1,
Time elapsed: 2.362 s - in org.apache.hadoop.fs.TestLocalFileSystem{color}
{color:#14892c}[INFO]{color}
{color:#14892c}[INFO] Results:{color}
{color:#14892c}[INFO]{color}
{color:#14892c}[WARNING] Tests run: 21, Failures: 0, Errors: 0, Skipped:
1{color}
> org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on
> Windows
> ---
>
> Key: HADOOP-15458
> URL: https://issues.apache.org/jira/browse/HADOOP-15458
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Xiao Liang
>Assignee: Xiao Liang
>Priority: Major
> Labels: windows
> Attachments: HADOOP-15458-branch-2.000.patch,
> HADOOP-15458-branch-2.001.patch, HADOOP-15458.000.patch,
> HADOOP-15458.001.patch
>
>
> In *org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder* a
> FSDataOutputStream object is unnecessarily created and not closed, which
> makes org.apache.hadoop.fs.TestLocalFileSystem#after fails to delete the
> folder on Windows.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15458) org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows
[ https://issues.apache.org/jira/browse/HADOOP-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Liang updated HADOOP-15458: Attachment: HADOOP-15458-branch-2.001.patch > org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on > Windows > --- > > Key: HADOOP-15458 > URL: https://issues.apache.org/jira/browse/HADOOP-15458 > Project: Hadoop Common > Issue Type: Bug >Reporter: Xiao Liang >Assignee: Xiao Liang >Priority: Major > Labels: windows > Attachments: HADOOP-15458-branch-2.000.patch, > HADOOP-15458-branch-2.001.patch, HADOOP-15458.000.patch, > HADOOP-15458.001.patch > > > In *org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder* a > FSDataOutputStream object is unnecessarily created and not closed, which > makes org.apache.hadoop.fs.TestLocalFileSystem#after fails to delete the > folder on Windows. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15481) Emit FairCallQueue stats as metrics
Erik Krogen created HADOOP-15481: Summary: Emit FairCallQueue stats as metrics Key: HADOOP-15481 URL: https://issues.apache.org/jira/browse/HADOOP-15481 Project: Hadoop Common Issue Type: Improvement Components: metrics, rpc-server Reporter: Erik Krogen Currently FairCallQueue has some statistics which are exported via JMX: the size of each queue, and the number of overflowed calls per queue. These are useful statistics to track over time to determine, for example, if queues need to be resized. We should emit them via the standard metrics system. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15465) Deprecate WinUtils#Symlinks by using native java code
[
https://issues.apache.org/jira/browse/HADOOP-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481229#comment-16481229
]
genericqa commented on HADOOP-15465:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
9s{color} | {color:green} root: The patch generated 0 new + 153 unchanged - 4
fixed = 153 total (was 157) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 15s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
44s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
6s{color} | {color:green} hadoop-yarn-server-tests in the patch passed. {color}
|
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}138m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15465 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924172/HADOOP-15465.v3.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient
[jira] [Commented] (HADOOP-15478) WASB: hflush() and hsync() regression
[
https://issues.apache.org/jira/browse/HADOOP-15478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481218#comment-16481218
]
genericqa commented on HADOOP-15478:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 59s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 14s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch generated
11 new + 10 unchanged - 0 fixed = 21 total (was 10) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 29s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
4s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15478 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924180/HADOOP-15478.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 0d3b6d81f4cb 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 89f5911 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14660/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14660/testReport/ |
| Max. process+thread count | 334 (vs. ulimit of 1) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14660/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
[jira] [Commented] (HADOOP-15458) org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows
[ https://issues.apache.org/jira/browse/HADOOP-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481194#comment-16481194 ] Íñigo Goiri commented on HADOOP-15458: -- [^HADOOP-15458.001.patch] LGTM. [~surmountian] do you need to post one for branch-2 or [^HADOOP-15458.001.patch] applies? Can you also confirm that this runs on Windows? > org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on > Windows > --- > > Key: HADOOP-15458 > URL: https://issues.apache.org/jira/browse/HADOOP-15458 > Project: Hadoop Common > Issue Type: Bug >Reporter: Xiao Liang >Assignee: Xiao Liang >Priority: Major > Labels: windows > Attachments: HADOOP-15458-branch-2.000.patch, HADOOP-15458.000.patch, > HADOOP-15458.001.patch > > > In *org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder* a > FSDataOutputStream object is unnecessarily created and not closed, which > makes org.apache.hadoop.fs.TestLocalFileSystem#after fails to delete the > folder on Windows. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15309) default maven in path under start-build-env.sh is the wrong one
[ https://issues.apache.org/jira/browse/HADOOP-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota reassigned HADOOP-15309: --- Assignee: Gabor Bota > default maven in path under start-build-env.sh is the wrong one > --- > > Key: HADOOP-15309 > URL: https://issues.apache.org/jira/browse/HADOOP-15309 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Allen Wittenauer >Assignee: Gabor Bota >Priority: Trivial > > PATH points to /usr/bin/mvn, should be /opt/maven/bin/mvn -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15458) org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows
[
https://issues.apache.org/jira/browse/HADOOP-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481180#comment-16481180
]
genericqa commented on HADOOP-15458:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 13s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 29m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 4s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
6s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}124m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15458 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924164/HADOOP-15458.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 595c8c0e77b2 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 89f5911 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14658/testReport/ |
| Max. process+thread count | 1717 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14658/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on
> Windows
>
[jira] [Updated] (HADOOP-15478) WASB: hflush() and hsync() regression
[ https://issues.apache.org/jira/browse/HADOOP-15478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marquardt updated HADOOP-15478: -- Status: Patch Available (was: Open) Submitting patch HADOOP-15478.001.patch > WASB: hflush() and hsync() regression > - > > Key: HADOOP-15478 > URL: https://issues.apache.org/jira/browse/HADOOP-15478 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.0.2, 2.9.0 >Reporter: Thomas Marquardt >Assignee: Thomas Marquardt >Priority: Major > Attachments: HADOOP-15478.001.patch > > > HADOOP-14520 introduced a regression in hflush() and hsync(). Previously, > for the default case where users upload data as block blobs, these were > no-ops. Unfortunately, HADOOP-14520 accidentally implemented hflush() and > hsync() by default, so any data buffered in the stream is immediately > uploaded to storage. This new behavior is undesirable, because block blobs > have a limit of 50,000 blocks. Spark users are now seeing failures due to > exceeding the block limit, since Spark frequently invokes hflush(). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15478) WASB: hflush() and hsync() regression
[ https://issues.apache.org/jira/browse/HADOOP-15478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481157#comment-16481157 ] Thomas Marquardt commented on HADOOP-15478: --- I've attached HADOOP-15478.001.patch with the fix and new test cases. The new testBlockBlobHFlush() and testBlockBlobHSync() tests fail without the fix, and pass with the fix. The new tests ensure correct behavior of flush, hflush, hsync, and close for the output streams used by the WASB driver. WASB test results against my Azure storage account are below: *$ mvn -T 1C -Dparallel-tests clean verify* Tests run: 241, Failures: 0, Errors: 0, Skipped: 11 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 Tests run: 444, Failures: 0, Errors: 0, Skipped: 55 Tests run: 126, Failures: 3, Errors: 0, Skipped: 10 Note 3 tests are failing in trunk. These are failing without my change, and I will look into them. *[ERROR] ITestNativeAzureFileSystemLive>NativeAzureFileSystemBaseTest.testFolderLastModifiedTime*: 651->NativeAzureFileSystemBaseTest.testModifiedTime:1492->Assert.assertTrue:41->Assert.fail:88 Modification time Fri May 18 20:21:56 UTC 2018 is not close to now: Fri May 18 20:21:40 UTC 2018 *[ERROR] ITestNativeAzureFileSystemLive>NativeAzureFileSystemBaseTest.testModifiedTimeForFile*: 626->NativeAzureFileSystemBaseTest.testModifiedTime:1492->Assert.assertTrue:41->Assert.fail:88 Modification time Fri May 18 20:23:31 UTC 2018 is not close to now: Fri May 18 20:23:15 UTC 2018 *[ERROR] ITestNativeAzureFileSystemLive>NativeAzureFileSystemBaseTest.testModifiedTimeForFolder*: 633->NativeAzureFileSystemBaseTest.testModifiedTime:1492->Assert.assertTrue:41->Assert.fail:88 Modification time Fri May 18 20:21:49 UTC 2018 is not close to now: Fri May 18 20:21:33 UTC 2018 > WASB: hflush() and hsync() regression > - > > Key: HADOOP-15478 > URL: https://issues.apache.org/jira/browse/HADOOP-15478 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 2.9.0, 3.0.2 >Reporter: Thomas Marquardt >Assignee: Thomas Marquardt >Priority: Major > Attachments: HADOOP-15478.001.patch > > > HADOOP-14520 introduced a regression in hflush() and hsync(). Previously, > for the default case where users upload data as block blobs, these were > no-ops. Unfortunately, HADOOP-14520 accidentally implemented hflush() and > hsync() by default, so any data buffered in the stream is immediately > uploaded to storage. This new behavior is undesirable, because block blobs > have a limit of 50,000 blocks. Spark users are now seeing failures due to > exceeding the block limit, since Spark frequently invokes hflush(). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15478) WASB: hflush() and hsync() regression
[ https://issues.apache.org/jira/browse/HADOOP-15478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marquardt updated HADOOP-15478: -- Attachment: HADOOP-15478.001.patch > WASB: hflush() and hsync() regression > - > > Key: HADOOP-15478 > URL: https://issues.apache.org/jira/browse/HADOOP-15478 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 2.9.0, 3.0.2 >Reporter: Thomas Marquardt >Assignee: Thomas Marquardt >Priority: Major > Attachments: HADOOP-15478.001.patch > > > HADOOP-14520 introduced a regression in hflush() and hsync(). Previously, > for the default case where users upload data as block blobs, these were > no-ops. Unfortunately, HADOOP-14520 accidentally implemented hflush() and > hsync() by default, so any data buffered in the stream is immediately > uploaded to storage. This new behavior is undesirable, because block blobs > have a limit of 50,000 blocks. Spark users are now seeing failures due to > exceeding the block limit, since Spark frequently invokes hflush(). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15465) Deprecate WinUtils#Symlinks by using native java code
[ https://issues.apache.org/jira/browse/HADOOP-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated HADOOP-15465: -- Attachment: HADOOP-15465.v3.patch > Deprecate WinUtils#Symlinks by using native java code > - > > Key: HADOOP-15465 > URL: https://issues.apache.org/jira/browse/HADOOP-15465 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Giovanni Matteo Fumarola >Priority: Major > Attachments: HADOOP-15465.v0.patch, HADOOP-15465.v0.proto.patch, > HADOOP-15465.v1.patch, HADOOP-15465.v2.patch, HADOOP-15465.v3.patch > > > Hadoop uses the shell to create symbolic links. Now that Hadoop relies on > Java 7+, we can deprecate all the shell code and rely on the Java APIs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15154) Abstract new method assertCapability for StreamCapabilities testing
[
https://issues.apache.org/jira/browse/HADOOP-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481049#comment-16481049
]
Hudson commented on HADOOP-15154:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14239 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14239/])
HADOOP-15154. Abstract new method assertCapability for (xiao: rev
89f59113927dd886f09d8fe2c05ff2cd5d1390c1)
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/TestCryptoStreams.java
> Abstract new method assertCapability for StreamCapabilities testing
> ---
>
> Key: HADOOP-15154
> URL: https://issues.apache.org/jira/browse/HADOOP-15154
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Xiao Chen
>Assignee: Zsolt Venczel
>Priority: Minor
> Fix For: 3.2.0
>
> Attachments: HADOOP-15154.01.patch, HADOOP-15154.02.patch,
> HADOOP-15154.03.patch
>
>
> From Steve's
> [comment|https://issues.apache.org/jira/browse/HADOOP-15149?focusedCommentId=16306806=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16306806]:
> bq. it'd have been cleaner for the asserts to have been one in a
> assertCapability(key, StreamCapabilities subject, bool outcome) and had it
> throw meaningful exceptions on a failure
> We can consider abstract such a method to a test util class and use it for
> all {{StreamCapabilities}} tests as needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15154) Abstract new method assertCapability for StreamCapabilities testing
[
https://issues.apache.org/jira/browse/HADOOP-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated HADOOP-15154:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.2.0
Status: Resolved (was: Patch Available)
Committed this to trunk!
Thank you [~zvenczel] for the contribution and [~ste...@apache.org] for the
review / idea.
> Abstract new method assertCapability for StreamCapabilities testing
> ---
>
> Key: HADOOP-15154
> URL: https://issues.apache.org/jira/browse/HADOOP-15154
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Xiao Chen
>Assignee: Zsolt Venczel
>Priority: Minor
> Fix For: 3.2.0
>
> Attachments: HADOOP-15154.01.patch, HADOOP-15154.02.patch,
> HADOOP-15154.03.patch
>
>
> From Steve's
> [comment|https://issues.apache.org/jira/browse/HADOOP-15149?focusedCommentId=16306806=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16306806]:
> bq. it'd have been cleaner for the asserts to have been one in a
> assertCapability(key, StreamCapabilities subject, bool outcome) and had it
> throw meaningful exceptions on a failure
> We can consider abstract such a method to a test util class and use it for
> all {{StreamCapabilities}} tests as needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15458) org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows
[ https://issues.apache.org/jira/browse/HADOOP-15458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Liang updated HADOOP-15458: Attachment: HADOOP-15458.001.patch > org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder fails on > Windows > --- > > Key: HADOOP-15458 > URL: https://issues.apache.org/jira/browse/HADOOP-15458 > Project: Hadoop Common > Issue Type: Bug >Reporter: Xiao Liang >Assignee: Xiao Liang >Priority: Major > Labels: windows > Attachments: HADOOP-15458-branch-2.000.patch, HADOOP-15458.000.patch, > HADOOP-15458.001.patch > > > In *org.apache.hadoop.fs.TestLocalFileSystem#testFSOutputStreamBuilder* a > FSDataOutputStream object is unnecessarily created and not closed, which > makes org.apache.hadoop.fs.TestLocalFileSystem#after fails to delete the > folder on Windows. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
[
https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480961#comment-16480961
]
Aaron Fabbri commented on HADOOP-15480:
---
I also reproduced this yesterday. Apologies for letting this through–not sure
why my testing for HADOOP-15420 didn't catch it.
BTW there is a hint in my description (see #2 there) for HADOOP-15420 on an
issue I found by inspection.
> AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
> ---
>
> Key: HADOOP-15480
> URL: https://issues.apache.org/jira/browse/HADOOP-15480
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.1.0
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
> testDiffCommand test fails with the following:
> {noformat}
> testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 8.059 s <<< FAILURE!
> java.lang.AssertionError:
> Mismatched metadata store outputs: MS D 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> expected:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]>
> but was:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
>
[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
[
https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HADOOP-15480:
Affects Version/s: 3.1.0
> AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
> ---
>
> Key: HADOOP-15480
> URL: https://issues.apache.org/jira/browse/HADOOP-15480
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.1.0
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
> testDiffCommand test fails with the following:
> {noformat}
> testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 8.059 s <<< FAILURE!
> java.lang.AssertionError:
> Mismatched metadata store outputs: MS D 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> expected:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]>
> but was:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}
--
[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
[
https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480772#comment-16480772
]
Steve Loughran commented on HADOOP-15480:
-
can you add version number? If its some internal branch, just use the closest
ASF release its built on. Thanks
> AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
> ---
>
> Key: HADOOP-15480
> URL: https://issues.apache.org/jira/browse/HADOOP-15480
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
> testDiffCommand test fails with the following:
> {noformat}
> testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 8.059 s <<< FAILURE!
> java.lang.AssertionError:
> Mismatched metadata store outputs: MS D 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> expected:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]>
> but was:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
>
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[
https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480742#comment-16480742
]
genericqa commented on HADOOP-14946:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 38s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
47s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-14946 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924120/HADOOP-14946.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 95b9b7f89c1d 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 6e99686 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14657/testReport/ |
| Max. process+thread count | 334 (vs. ulimit of 1) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14657/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> S3Guard testPruneCommandCLI can fail
>
>
> Key: HADOOP-14946
> URL:
[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
[
https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480718#comment-16480718
]
Sean Mackrory commented on HADOOP-15480:
Yes it does - and I think that's recent - I haven't seen that failure before
> AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
> ---
>
> Key: HADOOP-15480
> URL: https://issues.apache.org/jira/browse/HADOOP-15480
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
> testDiffCommand test fails with the following:
> {noformat}
> testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 8.059 s <<< FAILURE!
> java.lang.AssertionError:
> Mismatched metadata store outputs: MS D 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> expected:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]>
> but was:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
>
[jira] [Issue Comment Deleted] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
[
https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Mackrory updated HADOOP-15480:
---
Comment: was deleted
(was: Yes it does - and I think that's recent - I haven't seen that failure
before)
> AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
> ---
>
> Key: HADOOP-15480
> URL: https://issues.apache.org/jira/browse/HADOOP-15480
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
> testDiffCommand test fails with the following:
> {noformat}
> testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 8.059 s <<< FAILURE!
> java.lang.AssertionError:
> Mismatched metadata store outputs: MS D 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
> MSF 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> S3F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> MSF 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> expected:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]>
> but was:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
>
[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
[
https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HADOOP-15480:
Description:
When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
testDiffCommand test fails with the following:
{noformat}
testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
Time elapsed: 8.059 s <<< FAILURE!
java.lang.AssertionError:
Mismatched metadata store outputs: MS D 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
expected:<[
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]>
but was:<[
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at
org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{noformat}
was:
When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
testDiffCommand test fails with the following:
{noformat}
testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
Time elapsed: 8.059 s <<< FAILURE!
java.lang.AssertionError:
Mismatched metadata store outputs: MS D 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
MS F 100
[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
[
https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HADOOP-15480:
Description:
When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
testDiffCommand test fails with the following:
{noformat}
testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
Time elapsed: 8.059 s <<< FAILURE!
java.lang.AssertionError:
Mismatched metadata store outputs: MS D 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
expected:<[
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]>
but was:<[
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0, s
3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at
org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{noformat}
was:
When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
testDiffCommand test fails with the following:
{noformat}
testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
Time elapsed: 8.059 s <<< FAILURE!
java.lang.AssertionError:
Mismatched metadata store outputs: MS D 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
MS F 100
[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
[
https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HADOOP-15480:
Description:
When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
testDiffCommand test fails with the following:
{noformat}
testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
Time elapsed: 8.059 s <<< FAILURE!
java.lang.AssertionError:
Mismatched metadata store outputs: MS D 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
expected:<[
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]> but
was:<[s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at
org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{noformat}
was:
When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
testDiffCommand test fails with the following:
{noformat}
testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
Time elapsed: 8.059 s <<< FAILURE!
java.lang.AssertionError:
Mismatched metadata store outputs: MS D 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
MS F 100
[jira] [Created] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
Gabor Bota created HADOOP-15480:
---
Summary: AbstractS3GuardToolTestBase.testDiffCommand fails when
using dynamo
Key: HADOOP-15480
URL: https://issues.apache.org/jira/browse/HADOOP-15480
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Reporter: Gabor Bota
Assignee: Gabor Bota
When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
testDiffCommand test fails with the following:
{noformat}
testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
Time elapsed: 8.059 s <<< FAILURE!
java.lang.AssertionError:
Mismatched metadata store outputs: MS D 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
MS F 100
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
S3 F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
MS F 0
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
expected:<[s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]> but
was:<[s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at
org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[
https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HADOOP-14946:
Status: Patch Available (was: In Progress)
> S3Guard testPruneCommandCLI can fail
>
>
> Key: HADOOP-14946
> URL: https://issues.apache.org/jira/browse/HADOOP-14946
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-14946.001.patch
>
>
> The test of the S3Guard CLI prune can sometimes fail on parallel test runs.
> Assumption: it is the parallelism which is causing the problem
> {code}
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB
> testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 10.765 sec <<< FAILURE!
> java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0>
> at org.junit.Assert.fail(Assert.java:88)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[
https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480650#comment-16480650
]
Gabor Bota commented on HADOOP-14946:
-
I've also corrected a javadoc with my patch in AbstractS3GuardToolTestBase.
> S3Guard testPruneCommandCLI can fail
>
>
> Key: HADOOP-14946
> URL: https://issues.apache.org/jira/browse/HADOOP-14946
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-14946.001.patch
>
>
> The test of the S3Guard CLI prune can sometimes fail on parallel test runs.
> Assumption: it is the parallelism which is causing the problem
> {code}
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB
> testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 10.765 sec <<< FAILURE!
> java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0>
> at org.junit.Assert.fail(Assert.java:88)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[
https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota updated HADOOP-14946:
Attachment: HADOOP-14946.001.patch
> S3Guard testPruneCommandCLI can fail
>
>
> Key: HADOOP-14946
> URL: https://issues.apache.org/jira/browse/HADOOP-14946
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
> Attachments: HADOOP-14946.001.patch
>
>
> The test of the S3Guard CLI prune can sometimes fail on parallel test runs.
> Assumption: it is the parallelism which is causing the problem
> {code}
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB
> testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 10.765 sec <<< FAILURE!
> java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0>
> at org.junit.Assert.fail(Assert.java:88)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
[
https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480647#comment-16480647
]
Gabor Bota commented on HADOOP-14946:
-
I think increasing the timeout will solve this issue. Please test it, it passes
for me in {{eu-west-1}} with {{mvn -Dparallel-tests -DtestsThreadCount=8 clean
verify -Ds3guard -Ddynamo}}
> S3Guard testPruneCommandCLI can fail
>
>
> Key: HADOOP-14946
> URL: https://issues.apache.org/jira/browse/HADOOP-14946
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
>Assignee: Gabor Bota
>Priority: Major
>
> The test of the S3Guard CLI prune can sometimes fail on parallel test runs.
> Assumption: it is the parallelism which is causing the problem
> {code}
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB
> testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 10.765 sec <<< FAILURE!
> java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0>
> at org.junit.Assert.fail(Assert.java:88)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15479) s3guard bucket-info command to add a verify-property =
Steve Loughran created HADOOP-15479:
---
Summary: s3guard bucket-info command to add a verify-property
=
Key: HADOOP-15479
URL: https://issues.apache.org/jira/browse/HADOOP-15479
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: 3.1.0
Reporter: Steve Loughran
This is driven by me trying to test o whether a bucket has fault injection
enabled. You can see from the logs, but not from a shell script.
I want to be able to go
{code}
hadoop s3guard verify-property
fs.s3a.s3.client.factory.impl=org.apache.hadoop.fs.s3a.InconsistentS3ClientFactory
s3a://test-bucket/
{code}
and have the command return -1 if the property doesn't equal this value. This
lets me check that per-bucket and test run settings are propagating down. As it
is you need to look @ the logs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15154) Abstract new method assertCapability for StreamCapabilities testing
[
https://issues.apache.org/jira/browse/HADOOP-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480531#comment-16480531
]
genericqa commented on HADOOP-15154:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 22s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 29m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 41s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
13s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}132m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15154 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924082/HADOOP-15154.03.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 792c138d1b00 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 6e99686 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14656/testReport/ |
| Max. process+thread count | 1573 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14656/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Abstract new method assertCapability for StreamCapabilities testing
> ---
>
>
[jira] [Updated] (HADOOP-15154) Abstract new method assertCapability for StreamCapabilities testing
[
https://issues.apache.org/jira/browse/HADOOP-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsolt Venczel updated HADOOP-15154:
---
Attachment: HADOOP-15154.03.patch
> Abstract new method assertCapability for StreamCapabilities testing
> ---
>
> Key: HADOOP-15154
> URL: https://issues.apache.org/jira/browse/HADOOP-15154
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Xiao Chen
>Assignee: Zsolt Venczel
>Priority: Minor
> Attachments: HADOOP-15154.01.patch, HADOOP-15154.02.patch,
> HADOOP-15154.03.patch
>
>
> From Steve's
> [comment|https://issues.apache.org/jira/browse/HADOOP-15149?focusedCommentId=16306806=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16306806]:
> bq. it'd have been cleaner for the asserts to have been one in a
> assertCapability(key, StreamCapabilities subject, bool outcome) and had it
> throw meaningful exceptions on a failure
> We can consider abstract such a method to a test util class and use it for
> all {{StreamCapabilities}} tests as needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15478) WASB: hflush() and hsync() regression
Thomas Marquardt created HADOOP-15478: - Summary: WASB: hflush() and hsync() regression Key: HADOOP-15478 URL: https://issues.apache.org/jira/browse/HADOOP-15478 Project: Hadoop Common Issue Type: Bug Components: fs/azure Affects Versions: 3.0.2, 2.9.0 Reporter: Thomas Marquardt Assignee: Thomas Marquardt HADOOP-14520 introduced a regression in hflush() and hsync(). Previously, for the default case where users upload data as block blobs, these were no-ops. Unfortunately, HADOOP-14520 accidentally implemented hflush() and hsync() by default, so any data buffered in the stream is immediately uploaded to storage. This new behavior is undesirable, because block blobs have a limit of 50,000 blocks. Spark users are now seeing failures due to exceeding the block limit, since Spark frequently invokes hflush(). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15474) Rename properties introduced for
[
https://issues.apache.org/jira/browse/HADOOP-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480346#comment-16480346
]
genericqa commented on HADOOP-15474:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 27s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 26m 34s{color}
| {color:red} root generated 4 new + 1465 unchanged - 0 fixed = 1469 total (was
1465) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 15s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
10s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}121m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15474 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924059/HADOOP-15474.02.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux 5648aba40965 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 3159bff |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14655/artifact/out/diff-compile-javac-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14655/testReport/ |
| Max. process+thread count | 1532 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
[jira] [Commented] (HADOOP-15474) Rename properties introduced for
[
https://issues.apache.org/jira/browse/HADOOP-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480338#comment-16480338
]
Nanda kumar commented on HADOOP-15474:
--
[~zvenczel], thanks for updating the patch.
+1 (non-binding), looks good to me.
> Rename properties introduced for
> ---
>
> Key: HADOOP-15474
> URL: https://issues.apache.org/jira/browse/HADOOP-15474
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Affects Versions: 3.1.0
>Reporter: Nanda kumar
>Assignee: Zsolt Venczel
>Priority: Major
> Attachments: HADOOP-15474.01.patch, HADOOP-15474.02.patch
>
>
> HADOOP-15007 introduces the following two properties for tagging
> configuration properties
> * hadoop.system.tags
> * hadoop.custom.tags
> This sounds like {{tags}} fall under {{hadoop.system}} and {{hadoop.custom}}
> related properties, but what we really want to achieve here is to have two
> sub-division of {{tags}} namely {{system}} and {{custom}}
> For better readability, we can rename them as
> * hadoop.tags.system
> * hadoop.tags.custom
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15477) Make unjar in RunJar overrideable
[ https://issues.apache.org/jira/browse/HADOOP-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480285#comment-16480285 ] Johan Gustavsson commented on HADOOP-15477: --- [~ajisakaa] could I ask you to take a look at this trivial patch or reassign it to whoever would be the right person to review it? Reason for no test is due to it's simplicity. > Make unjar in RunJar overrideable > - > > Key: HADOOP-15477 > URL: https://issues.apache.org/jira/browse/HADOOP-15477 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.8.3, 2.9.1, 2.7.6, 3.0.2 >Reporter: Johan Gustavsson >Priority: Trivial > Attachments: HADOOP-15477.001.patch, HADOOP-15477.002.patch > > > Currently Hadoop's RunJar will unjar the jar provided and look for any jars > inside and add them to the classpath. Since most deployments doesn't use jar > in jar, but rather uberjars this could be rather time consuming at times and > can cause issues related to over consumption of inodes, for something that is > in many cases is not used. > For that purpose there should be an env variable to disable this behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15477) Make unjar in RunJar overrideable
[
https://issues.apache.org/jira/browse/HADOOP-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480281#comment-16480281
]
genericqa commented on HADOOP-15477:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 29m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 25s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
28s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}126m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15477 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924050/HADOOP-15477.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 6a73ff87edd7 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 3159bff |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14653/testReport/ |
| Max. process+thread count | 1629 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14653/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Make unjar in RunJar
[jira] [Commented] (HADOOP-15154) Abstract new method assertCapability for StreamCapabilities testing
[
https://issues.apache.org/jira/browse/HADOOP-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480220#comment-16480220
]
Xiao Chen commented on HADOOP-15154:
Thanks Zsolt for the patch and Steve for the review.
+1 pending checkstyle fix
> Abstract new method assertCapability for StreamCapabilities testing
> ---
>
> Key: HADOOP-15154
> URL: https://issues.apache.org/jira/browse/HADOOP-15154
> Project: Hadoop Common
> Issue Type: Improvement
> Components: test
>Reporter: Xiao Chen
>Assignee: Zsolt Venczel
>Priority: Minor
> Attachments: HADOOP-15154.01.patch, HADOOP-15154.02.patch
>
>
> From Steve's
> [comment|https://issues.apache.org/jira/browse/HADOOP-15149?focusedCommentId=16306806=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16306806]:
> bq. it'd have been cleaner for the asserts to have been one in a
> assertCapability(key, StreamCapabilities subject, bool outcome) and had it
> throw meaningful exceptions on a failure
> We can consider abstract such a method to a test util class and use it for
> all {{StreamCapabilities}} tests as needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15474) Rename properties introduced for
[
https://issues.apache.org/jira/browse/HADOOP-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480219#comment-16480219
]
Zsolt Venczel commented on HADOOP-15474:
Thanks [~nandakumar131] for taking a look!
I applied the changes you suggested in the latest patch.
> Rename properties introduced for
> ---
>
> Key: HADOOP-15474
> URL: https://issues.apache.org/jira/browse/HADOOP-15474
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Affects Versions: 3.1.0
>Reporter: Nanda kumar
>Assignee: Zsolt Venczel
>Priority: Major
> Attachments: HADOOP-15474.01.patch, HADOOP-15474.02.patch
>
>
> HADOOP-15007 introduces the following two properties for tagging
> configuration properties
> * hadoop.system.tags
> * hadoop.custom.tags
> This sounds like {{tags}} fall under {{hadoop.system}} and {{hadoop.custom}}
> related properties, but what we really want to achieve here is to have two
> sub-division of {{tags}} namely {{system}} and {{custom}}
> For better readability, we can rename them as
> * hadoop.tags.system
> * hadoop.tags.custom
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15474) Rename properties introduced for
[
https://issues.apache.org/jira/browse/HADOOP-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsolt Venczel updated HADOOP-15474:
---
Attachment: HADOOP-15474.02.patch
> Rename properties introduced for
> ---
>
> Key: HADOOP-15474
> URL: https://issues.apache.org/jira/browse/HADOOP-15474
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Affects Versions: 3.1.0
>Reporter: Nanda kumar
>Assignee: Zsolt Venczel
>Priority: Major
> Attachments: HADOOP-15474.01.patch, HADOOP-15474.02.patch
>
>
> HADOOP-15007 introduces the following two properties for tagging
> configuration properties
> * hadoop.system.tags
> * hadoop.custom.tags
> This sounds like {{tags}} fall under {{hadoop.system}} and {{hadoop.custom}}
> related properties, but what we really want to achieve here is to have two
> sub-division of {{tags}} namely {{system}} and {{custom}}
> For better readability, we can rename them as
> * hadoop.tags.system
> * hadoop.tags.custom
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15474) Rename properties introduced for
[
https://issues.apache.org/jira/browse/HADOOP-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsolt Venczel updated HADOOP-15474:
---
Attachment: (was: HADOOP-15474.02.patch)
> Rename properties introduced for
> ---
>
> Key: HADOOP-15474
> URL: https://issues.apache.org/jira/browse/HADOOP-15474
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Affects Versions: 3.1.0
>Reporter: Nanda kumar
>Assignee: Zsolt Venczel
>Priority: Major
> Attachments: HADOOP-15474.01.patch
>
>
> HADOOP-15007 introduces the following two properties for tagging
> configuration properties
> * hadoop.system.tags
> * hadoop.custom.tags
> This sounds like {{tags}} fall under {{hadoop.system}} and {{hadoop.custom}}
> related properties, but what we really want to achieve here is to have two
> sub-division of {{tags}} namely {{system}} and {{custom}}
> For better readability, we can rename them as
> * hadoop.tags.system
> * hadoop.tags.custom
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15474) Rename properties introduced for
[
https://issues.apache.org/jira/browse/HADOOP-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsolt Venczel updated HADOOP-15474:
---
Attachment: HADOOP-15474.02.patch
> Rename properties introduced for
> ---
>
> Key: HADOOP-15474
> URL: https://issues.apache.org/jira/browse/HADOOP-15474
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Affects Versions: 3.1.0
>Reporter: Nanda kumar
>Assignee: Zsolt Venczel
>Priority: Major
> Attachments: HADOOP-15474.01.patch
>
>
> HADOOP-15007 introduces the following two properties for tagging
> configuration properties
> * hadoop.system.tags
> * hadoop.custom.tags
> This sounds like {{tags}} fall under {{hadoop.system}} and {{hadoop.custom}}
> related properties, but what we really want to achieve here is to have two
> sub-division of {{tags}} namely {{system}} and {{custom}}
> For better readability, we can rename them as
> * hadoop.tags.system
> * hadoop.tags.custom
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
