[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13966: --- Release Note: For running integration tests, the command "mvn verify -Dscale -Ds3guard -Ddynamodblocal -q" will be a faster DDB without bills to pay. In-memory instance will be started automatically. For tests only. > Add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran >Assignee: Mingliang Liu > Fix For: HADOOP-13345 > > Attachments: HADOOP-13966-HADOOP-13345.000.patch, > HADOOP-13966-HADOOP-13345.001.patch, HADOOP-13966-HADOOP-13345.002.patch, > HADOOP-13966-HADOOP-13345.003.patch > > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13966: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HADOOP-13345 Release Note: (was: For running integration tests, the command "mvn verify -Dscale -Ds3guard -Ddynamodblocal -q" will be a faster DDB without bills to pay. In-memory instance will be started automatically. For tests only.) Status: Resolved (was: Patch Available) Thanks [~ste...@apache.org] for reviewing and committing. I revert the v2 patch and committed the v3 patch to the feature branch. > Add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran >Assignee: Mingliang Liu > Fix For: HADOOP-13345 > > Attachments: HADOOP-13966-HADOOP-13345.000.patch, > HADOOP-13966-HADOOP-13345.001.patch, HADOOP-13966-HADOOP-13345.002.patch, > HADOOP-13966-HADOOP-13345.003.patch > > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13966: --- Release Note: For running integration tests, the command "mvn verify -Dscale -Ds3guard -Ddynamodblocal -q" will be a faster DDB without bills to pay. In-memory instance will be started automatically. For tests only. > Add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran >Assignee: Mingliang Liu > Attachments: HADOOP-13966-HADOOP-13345.000.patch, > HADOOP-13966-HADOOP-13345.001.patch, HADOOP-13966-HADOOP-13345.002.patch, > HADOOP-13966-HADOOP-13345.003.patch > > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test
[
https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-13966:
---
Attachment: HADOOP-13966-HADOOP-13345.003.patch
The v3 patch fixes failing test {{ITestS3GuardToolDynamoDB}} against
DynamoDBLocal.
> Add ability to start DDB local server in every test
> ---
>
> Key: HADOOP-13966
> URL: https://issues.apache.org/jira/browse/HADOOP-13966
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: HADOOP-13345
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HADOOP-13966-HADOOP-13345.000.patch,
> HADOOP-13966-HADOOP-13345.001.patch, HADOOP-13966-HADOOP-13345.002.patch,
> HADOOP-13966-HADOOP-13345.003.patch
>
>
> the local in memory DDB starts up in only 2+ seconds, so we have to reason to
> not use it in all our integration tests, if we add a switch to do this
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13966: --- Attachment: HADOOP-13966-HADOOP-13345.002.patch > Add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran >Assignee: Mingliang Liu > Attachments: HADOOP-13966-HADOOP-13345.000.patch, > HADOOP-13966-HADOOP-13345.001.patch, HADOOP-13966-HADOOP-13345.002.patch > > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test
[
https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-13966:
---
Attachment: HADOOP-13966-HADOOP-13345.001.patch
The v1 patch address all Steve's comments except #3.
# I found the native libs are loaded as read-only, so I think sharing between
parallel tests should be fine?
# In the maven pom.xml we have to check if how {{test.build.data}} is set and
copy the native-libs accordingly. However, it can be unset, or overridden in
the maven-surefire-plugin. This seems not very straightforward to check this in
pom.xml
Existing code for copying native-libs:
{code:title=hadoop-aws/pom.xml}
copy
test-compile
copy-dependencies
test
so,dll,dylib
target/native-libs
{code}
I'll discuss with Steve offline and post comments here.
> Add ability to start DDB local server in every test
> ---
>
> Key: HADOOP-13966
> URL: https://issues.apache.org/jira/browse/HADOOP-13966
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: HADOOP-13345
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HADOOP-13966-HADOOP-13345.000.patch,
> HADOOP-13966-HADOOP-13345.001.patch
>
>
> the local in memory DDB starts up in only 2+ seconds, so we have to reason to
> not use it in all our integration tests, if we add a switch to do this
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13966: --- Status: Patch Available (was: Open) > Add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran >Assignee: Mingliang Liu > Attachments: HADOOP-13966-HADOOP-13345.000.patch, > HADOOP-13966-HADOOP-13345.001.patch > > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13966) add ability to start DDB local server in every test
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13966: --- Attachment: HADOOP-13966-HADOOP-13345.000.patch V0 patch is a starting point. Open question is how to reuse the singleton instance and when to stop it. > add ability to start DDB local server in every test > --- > > Key: HADOOP-13966 > URL: https://issues.apache.org/jira/browse/HADOOP-13966 > Project: Hadoop Common > Issue Type: Sub-task > Components: test >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran > Attachments: HADOOP-13966-HADOOP-13345.000.patch > > > the local in memory DDB starts up in only 2+ seconds, so we have to reason to > not use it in all our integration tests, if we add a switch to do this -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
