[jira] [Commented] (BEAM-1176) Make our test suites use @Rule TestPipeline

2016-12-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764973#comment-15764973
 ] 

ASF GitHub Bot commented on BEAM-1176:
--

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1664


> Make our test suites use @Rule TestPipeline
> ---
>
> Key: BEAM-1176
> URL: https://issues.apache.org/jira/browse/BEAM-1176
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Stas Levin
>Priority: Minor
>
> Now that [~staslev] has made {{TestPipeline}} a JUnit rule that performs 
> useful sanity checks, we should port all of our tests to it so that they set 
> a good example for users. Maybe we'll even catch some straggling tests with 
> errors :-)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-1176) Make our test suites use @Rule TestPipeline

2016-12-20 Thread Stas Levin (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764919#comment-15764919
 ] 

Stas Levin commented on BEAM-1176:
--

Sounds good, enter [BEAM-1186|https://issues.apache.org/jira/browse/BEAM-1186].

> Make our test suites use @Rule TestPipeline
> ---
>
> Key: BEAM-1176
> URL: https://issues.apache.org/jira/browse/BEAM-1176
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Stas Levin
>Priority: Minor
>
> Now that [~staslev] has made {{TestPipeline}} a JUnit rule that performs 
> useful sanity checks, we should port all of our tests to it so that they set 
> a good example for users. Maybe we'll even catch some straggling tests with 
> errors :-)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-1176) Make our test suites use @Rule TestPipeline

2016-12-20 Thread Luke Cwik (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764778#comment-15764778
 ] 

Luke Cwik commented on BEAM-1176:
-

I took a look at AvroIOGeneratedClassTest, ApproximateUniqueTest, SampleTest 
and BigtableIOTest. It seemed as though all multi-pipeline cases were just 
different variants of the same pipeline. It seems as though if these tests were 
broken up into multiple tests or better yet a set of paramemterized tests 
(https://github.com/Pragmatists/junitparams), we could use the test rule.

> Make our test suites use @Rule TestPipeline
> ---
>
> Key: BEAM-1176
> URL: https://issues.apache.org/jira/browse/BEAM-1176
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Stas Levin
>Priority: Minor
>
> Now that [~staslev] has made {{TestPipeline}} a JUnit rule that performs 
> useful sanity checks, we should port all of our tests to it so that they set 
> a good example for users. Maybe we'll even catch some straggling tests with 
> errors :-)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-1176) Make our test suites use @Rule TestPipeline

2016-12-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764731#comment-15764731
 ] 

ASF GitHub Bot commented on BEAM-1176:
--

GitHub user staslev opened a pull request:

https://github.com/apache/incubator-beam/pull/1664

[BEAM-1176] Migrating test to use TestPipeline as a JUnit rule

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/staslev/incubator-beam 
BEAM-1176-migrating-to-TestPipeline-rule

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1664.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1664


commit 83068990d803edabee7aefa32de9542f84d087ac
Author: Stas Levin 
Date:   2016-12-18T16:25:33Z

Migrated the beam-examples-java module to TestPipeline as a JUnit rule.

commit ef9cabfc8dbc95d6f7748404740751fec5c9a17c
Author: Stas Levin 
Date:   2016-12-18T16:38:11Z

Migrated the beam-examples-java8 module to TestPipeline as a JUnit rule.

commit 6b68d7989d0a4fd591a10b49d40f9d77d29d3ac2
Author: Stas Levin 
Date:   2016-12-18T16:51:31Z

Migrated the beam-runners-core module to TestPipeline as a JUnit rule.

commit d95030560baa00e765f481fed256b9ad7ab00e53
Author: Stas Levin 
Date:   2016-12-19T08:20:16Z

Migrated the beam-runners-direct-java module to TestPipeline as a JUnit 
rule.

commit 2ea56f452f3fc19920e2b53f7effdb77e5774e76
Author: Stas Levin 
Date:   2016-12-19T21:54:47Z

Migrated the beam-sdks-java-core module to TestPipeline as a JUnit rule.
Plus, fixed some checkstyle errors from previous modules' migration.

commit 5e23bee64d0f186071cb90f95293abcfcbfb5250
Author: Stas Levin 
Date:   2016-12-19T22:01:31Z

Migrated the beam-sdks-java-extensions-join-library module to TestPipeline 
as a JUnit rule.

commit 9ae205550f222501fdc7de8d89666b98ab0f5620
Author: Stas Levin 
Date:   2016-12-20T07:54:57Z

Migrated the beam-sdks-java-extensions-sorter module to TestPipeline as a 
JUnit rule.

commit 0bf119d677112d5ed7f15623f86e5478ce949b13
Author: Stas Levin 
Date:   2016-12-20T11:26:07Z

Migrated the beam-sdks-java-io-google-cloud-platform module to TestPipeline 
as a JUnit rule.

commit d6207df93712fc53e3921f2da1ae42a86dbd9696
Author: Stas Levin 
Date:   2016-12-20T15:26:51Z

Migrated the beam-sdks-java-io-jdbc module to TestPipeline as a JUnit rule.

commit e3c5841a017fd71ac04c8550964753eb1a5fa802
Author: Stas Levin 
Date:   2016-12-20T15:31:23Z

Migrated the beam-sdks-java-io-jms module to TestPipeline as a JUnit rule.

commit 75020fbc235e4e1c57a4efd12c6a70ffcc763205
Author: Stas Levin 
Date:   2016-12-20T15:38:38Z

Migrated the beam-sdks-java-io-kafka module to TestPipeline as a JUnit rule.

commit 316ddcac2e8eadd778a20426e7e3cc746adbc767
Author: Stas Levin 
Date:   2016-12-20T15:44:15Z

Migrated the beam-sdks-java-io-kinesis module to TestPipeline as a JUnit 
rule.

commit 1e6390c917bc0365c440d505dc87e1ea6b13fe32
Author: Stas Levin 
Date:   2016-12-20T16:09:30Z

Migrated the beam-sdks-java-io-mongodb module to TestPipeline as a JUnit 
rule.

commit 51d38973076d9ed03bba2c38fe2f70b0ce17f6d4
Author: Stas Levin 
Date:   2016-12-20T16:57:57Z

Migrated the beam-sdks-java-io-java8tests module to TestPipeline as a JUnit 
rule + fixed WithTimestampsJava8Test.withTimestampsLambdaShouldApplyTimestamps.

commit 07a46f1a998f49b275d9639c92a0461d68803b77
Author: Stas Levin 
Date:   2016-12-20T17:15:44Z

Fixed checkstyle issues.




> Make our test suites use @Rule TestPipeline
> ---
>
> Key: BEAM-1176
> URL: https://issues.apache.org/jira/browse/BEAM-1176
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>

[jira] [Commented] (BEAM-1176) Make our test suites use @Rule TestPipeline

2016-12-20 Thread Stas Levin (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763964#comment-15763964
 ] 

Stas Levin commented on BEAM-1176:
--

I've started migrating the tests to using the {{TestPipeline}} as a JUnit rule. 
There's quite a few of places to change so this might take a while longer.

>From the migration I've done so far, there's a particularly notable pattern we 
>don't currently support in the {{TestPipeline}} rule - having multiple 
>pipeline instances within a single test method. 
This goes beyond supporting multiple {{run}} invocations on the same 
{{TestPipeline}} rule instance, since such tests make assumption about their 
internal pipeline instances being isolated. This in turn translates to their 
{{PTransform}} names often being unstable, and/or dangling pipeline nodes which 
are acceptable since error handling is performed inside dedicated try/catch 
blocks inside the test itself.

Examples for such cases can be found the following tests:
*  {{AvroIOGeneratedClassTest#runTestRead}}
*  
{{ApproximateUniqueTest#runApproximateUniqueWithDuplicates}},{{ApproximateUniqueTest#runApproximateUniqueWithSkewedDistributions}}
*  {{SampleTest#runPickAnyTest}}
*  {{BigtableIOTest#runReadTest}}

For now I'm refraining from migrating these particular test cases by preserving 
direct calls to {{TestPipeline.create()}} in places other than the {{@Rule}} 
declaration itself, which ideally I guess we should avoid.

Any thoughts?

> Make our test suites use @Rule TestPipeline
> ---
>
> Key: BEAM-1176
> URL: https://issues.apache.org/jira/browse/BEAM-1176
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Stas Levin
>Priority: Minor
>
> Now that [~staslev] has made {{TestPipeline}} a JUnit rule that performs 
> useful sanity checks, we should port all of our tests to it so that they set 
> a good example for users. Maybe we'll even catch some straggling tests with 
> errors :-)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)