[jira] [Commented] (HADOOP-13966) Add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940700#comment-15940700 ] Steve Loughran commented on HADOOP-13966: - OK, +1 on this (I'd already committed it), sorry for getting the wrong one in. Mingliang, on the basis that you know what you are doing, I'll delegate the rollback to you > Add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran >Assignee: Mingliang Liu > Attachments: HADOOP-13966-HADOOP-13345.000.patch, > HADOOP-13966-HADOOP-13345.001.patch, HADOOP-13966-HADOOP-13345.002.patch, > HADOOP-13966-HADOOP-13345.003.patch > > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13966) Add ability to start DDB local server in every test
[
https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937461#comment-15937461
]
Mingliang Liu commented on HADOOP-13966:
{code}
$ mvn verify -Dit.test='ITestS3A*,ITestS3Guard*,ITestDynamo*' -Dtest=none
-Dscale -Ds3guard -Ddynamodblocal -q
Results :
Tests run: 357, Failures: 0, Errors: 0, Skipped: 16
$ mvn verify -Dit.test='ITestS3A*,ITestS3Guard*,ITestDynamo*' -Dtest=none
-Dscale -Ds3guard -Ddynamo -q
Results :
Tests run: 357, Failures: 0, Errors: 0, Skipped: 16
{code}
> Add ability to start DDB local server in every test
> ---
>
> Key: HADOOP-13966
> URL: https://issues.apache.org/jira/browse/HADOOP-13966
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: HADOOP-13345
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HADOOP-13966-HADOOP-13345.000.patch,
> HADOOP-13966-HADOOP-13345.001.patch, HADOOP-13966-HADOOP-13345.002.patch,
> HADOOP-13966-HADOOP-13345.003.patch
>
>
> the local in memory DDB starts up in only 2+ seconds, so we have to reason to
> not use it in all our integration tests, if we add a switch to do this
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13966) Add ability to start DDB local server in every test
[
https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937179#comment-15937179
]
Hadoop QA commented on HADOOP-13966:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
37s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
40s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-13966 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12860007/HADOOP-13966-HADOOP-13345.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux a1763ea4942d 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HADOOP-13345 / fcc8f65 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11883/testReport/ |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11883/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add ability to start DDB local server in every test
> ---
>
> Key: HADOOP-13966
> URL: https://issues.apache.org/jira/browse/HADOOP-13966
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: HADOOP-13345
>Reporter: Steve Loughran
>Assignee:
[jira] [Commented] (HADOOP-13966) Add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936616#comment-15936616 ] Steve Loughran commented on HADOOP-13966: - OK, I'm confused about the native libs; I thought it was something set up in the java code, the way things like leveldb to, but I see not > Add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran >Assignee: Mingliang Liu > Attachments: HADOOP-13966-HADOOP-13345.000.patch, > HADOOP-13966-HADOOP-13345.001.patch, HADOOP-13966-HADOOP-13345.002.patch > > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13966) Add ability to start DDB local server in every test
[
https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15935830#comment-15935830
]
Hadoop QA commented on HADOOP-13966:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
0s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 13s{color} | {color:orange} hadoop-tools/hadoop-aws: The patch generated 4
new + 10 unchanged - 1 fixed = 14 total (was 11) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HADOOP-13966 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12859870/HADOOP-13966-HADOOP-13345.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux 400054ab3f10 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HADOOP-13345 / fcc8f65 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11871/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-aws.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11871/testReport/ |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11871/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add ability to start DDB local server in every test
> ---
>
> Key: HADOOP-13966
> URL:
[jira] [Commented] (HADOOP-13966) add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15934842#comment-15934842 ] Steve Loughran commented on HADOOP-13966: - # don't worry about the logout, JVM termination will do that # do skip setting the sql lite sysprop if already set # we need to make sure that the sqllist property is unique for multiple JVMs running in parallel. That could be done by using the system property test.build.data , which is customised in the maven parallel test runner, falling back to "target" if unset. # oh, and use the OS specific "/" path separator character > add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran > Attachments: HADOOP-13966-HADOOP-13345.000.patch > > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13966) add ability to start DDB local server in every test
[
https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929710#comment-15929710
]
Steve Loughran commented on HADOOP-13966:
-
yeah, wondering how best to do it.
How about we have a JVM-wide singleton, in the {{AbstractS3ATestBase}} for
this: if the DDB isn't running, start it. If it is, leave alone.
> add ability to start DDB local server in every test
> ---
>
> Key: HADOOP-13966
> URL: https://issues.apache.org/jira/browse/HADOOP-13966
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: test
>Affects Versions: HADOOP-13345
>Reporter: Steve Loughran
>
> the local in memory DDB starts up in only 2+ seconds, so we have to reason to
> not use it in all our integration tests, if we add a switch to do this
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13966) add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927097#comment-15927097 ] Mingliang Liu commented on HADOOP-13966: So right now we use DDBLocal only in uni tests - the only one by now is TestDynamoDBMetadataStore. For integration tests, we always use the real DDB web service. Are you suggesting we make it possible to run the integration tests against DDBLocal? I'm +1 on the idea for faster/cheaper debugging. Nice to have. We may need a specialized DDBClientFactory class for connecting a standalone DDBLocal server for that. > add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
