[jira] [Commented] (BEAM-613) SimpleStreamingWordCountTest tests only a single batch

2016-09-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510728#comment-15510728
 ] 

ASF GitHub Bot commented on BEAM-613:
-

GitHub user staslev opened a pull request:

https://github.com/apache/incubator-beam/pull/982

[BEAM-613] Revised SimpleStreamingWordCountTest to test multiple batches

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

R: @amitsela 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/staslev/incubator-beam 
BEAM-613-Improve-SimpleStreamingWordCountTest

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/982.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #982


commit b2d84ae0b97220bd2d187d17db03a47c41128365
Author: Stas Levin 
Date:   2016-09-05T15:22:59Z

Revised the test to test multiple batches




> SimpleStreamingWordCountTest tests only a single batch
> --
>
> Key: BEAM-613
> URL: https://issues.apache.org/jira/browse/BEAM-613
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-spark
>Reporter: Stas Levin
>Assignee: Stas Levin
>
> {{org.apache.beam.runners.spark.translation.streaming.SimpleStreamingWordCountTest}}
>  aims to test a simple Spark streaming job, but only tests a single batch, 
> which is uncharacteristic of an actual (even simple) streaming job, usually 
> consisting of multiple batches.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-613) SimpleStreamingWordCountTest tests only a single batch

2016-09-12 Thread Stas Levin (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483768#comment-15483768
 ] 

Stas Levin commented on BEAM-613:
-

Pending [pull #909|https://github.com/apache/incubator-beam/pull/909].

> SimpleStreamingWordCountTest tests only a single batch
> --
>
> Key: BEAM-613
> URL: https://issues.apache.org/jira/browse/BEAM-613
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-spark
>Reporter: Stas Levin
>Assignee: Stas Levin
>
> {{org.apache.beam.runners.spark.translation.streaming.SimpleStreamingWordCountTest}}
>  aims to test a simple Spark streaming job, but only tests a single batch, 
> which is uncharacteristic of an actual (even simple) streaming job, usually 
> consisting of multiple batches.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-613) SimpleStreamingWordCountTest tests only a single batch

2016-09-01 Thread Stas Levin (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455424#comment-15455424
 ] 

Stas Levin commented on BEAM-613:
-

This improvement will helps detecting issues such as BEAM-15, where the 
StorageLevel inconsistency between DStream and RDD comes into play only when 
there are multiple batches.

> SimpleStreamingWordCountTest tests only a single batch
> --
>
> Key: BEAM-613
> URL: https://issues.apache.org/jira/browse/BEAM-613
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-spark
>Reporter: Stas Levin
>Assignee: Amit Sela
>
> {{org.apache.beam.runners.spark.translation.streaming.SimpleStreamingWordCountTest}}
>  aims to test a simple Spark streaming job, but only tests a single batch, 
> which is uncharacteristic of an actual (even simple) streaming job, usually 
> consisting of multiple batches.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)