[jira] [Updated] (BEAM-613) SimpleStreamingWordCountTest tests only a single batch

2016-09-01 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-613: Description: {{org.apache.beam.runners.spark.translation.streaming.SimpleStreamingWordCountTest}} aims to

[jira] [Created] (BEAM-613) SimpleStreamingWordCountTest tests only a single batch

2016-09-01 Thread Stas Levin (JIRA)
Stas Levin created BEAM-613: --- Summary: SimpleStreamingWordCountTest tests only a single batch Key: BEAM-613 URL: https://issues.apache.org/jira/browse/BEAM-613 Project: Beam Issue Type:

[jira] [Comment Edited] (BEAM-613) SimpleStreamingWordCountTest tests only a single batch

2016-09-01 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455424#comment-15455424 ] Stas Levin edited comment on BEAM-613 at 9/1/16 1:42 PM: - This improvement will help

[jira] [Commented] (BEAM-613) SimpleStreamingWordCountTest tests only a single batch

2016-09-01 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455424#comment-15455424 ] Stas Levin commented on BEAM-613: - This improvement will helps detecting issues such as BEAM-15, where the

[jira] [Resolved] (BEAM-628) Sink class org.apache.beam.runners.spark.aggregators.metrics.sink.GraphiteSink cannot be instantiated

2016-09-12 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin resolved BEAM-628. - Resolution: Fixed > Sink class > org.apache.beam.runners.spark.aggregators.metrics.sink.GraphiteSink

[jira] [Updated] (BEAM-628) Sink class org.apache.beam.runners.spark.aggregators.metrics.sink.GraphiteSink cannot be instantiated

2016-09-12 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-628: Fix Version/s: 0.3.0-incubating > Sink class >

[jira] [Commented] (BEAM-613) SimpleStreamingWordCountTest tests only a single batch

2016-09-12 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483768#comment-15483768 ] Stas Levin commented on BEAM-613: - Pending [pull #909|https://github.com/apache/incubator-beam/pull/909]. >

[jira] [Created] (BEAM-628) Sink class org.apache.beam.runners.spark.aggregators.metrics.sink.GraphiteSink cannot be instantiated

2016-09-12 Thread Stas Levin (JIRA)
Stas Levin created BEAM-628: --- Summary: Sink class org.apache.beam.runners.spark.aggregators.metrics.sink.GraphiteSink cannot be instantiated Key: BEAM-628 URL: https://issues.apache.org/jira/browse/BEAM-628

[jira] [Created] (BEAM-637) WindowedValue$ValueInGlobalWindow is not serializable when using JavaSerializer instead of Kryo

2016-09-19 Thread Stas Levin (JIRA)
Stas Levin created BEAM-637: --- Summary: WindowedValue$ValueInGlobalWindow is not serializable when using JavaSerializer instead of Kryo Key: BEAM-637 URL: https://issues.apache.org/jira/browse/BEAM-637

[jira] [Updated] (BEAM-637) Support Java serialisation (via JavaSerialiser)

2016-09-23 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-637: Issue Type: Improvement (was: Bug) > Support Java serialisation (via JavaSerialiser) >

[jira] [Updated] (BEAM-637) WindowedValue$ValueInGlobalWindow is not serializable when using JavaSerializer instead of Kryo

2016-09-23 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-637: Priority: Minor (was: Major) > WindowedValue$ValueInGlobalWindow is not serializable when using >

[jira] [Commented] (BEAM-637) WindowedValue$ValueInGlobalWindow is not serializable when using JavaSerializer instead of Kryo

2016-09-23 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516207#comment-15516207 ] Stas Levin commented on BEAM-637: - [~amitsela] I agree, we can make it about supporting Java serialisation

[jira] [Updated] (BEAM-637) Support Java serialisation (via JavaSerialiser)

2016-09-23 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-637: Summary: Support Java serialisation (via JavaSerialiser) (was: WindowedValue$ValueInGlobalWindow is not

[jira] [Updated] (BEAM-613) Add windwoding use cases to SimpleStreamingWordCountTest

2016-09-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-613: Summary: Add windwoding use cases to SimpleStreamingWordCountTest (was: SimpleStreamingWordCountTest tests

[jira] [Updated] (BEAM-613) Add windwoding use cases to SimpleStreamingWordCountTest

2016-09-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-613: Description: {{org.apache.beam.runners.spark.translation.streaming.SimpleStreamingWordCountTest}} should

[jira] [Updated] (BEAM-613) SimpleStreamingWordCountTest does not properly test fixed windows

2016-09-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-613: Summary: SimpleStreamingWordCountTest does not properly test fixed windows (was: Add windwoding use cases to

[jira] [Updated] (BEAM-613) SimpleStreamingWordCountTest does not properly test fixed windows

2016-09-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-613: Description: {{org.apache.beam.runners.spark.translation.streaming.SimpleStreamingWordCountTest}} does not

[jira] [Created] (BEAM-669) SimpleStreamingWordCountTest does not have a test for sliding windows

2016-09-22 Thread Stas Levin (JIRA)
Stas Levin created BEAM-669: --- Summary: SimpleStreamingWordCountTest does not have a test for sliding windows Key: BEAM-669 URL: https://issues.apache.org/jira/browse/BEAM-669 Project: Beam Issue

[jira] [Commented] (BEAM-579) Integrate NamedAggregators into Spark's sink system

2016-08-23 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432450#comment-15432450 ] Stas Levin commented on BEAM-579: - I've started working on it and should have a patch pretty soon. >

[jira] [Updated] (BEAM-579) Integrate NamedAggregators into Spark's sink system

2016-08-23 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-579: Priority: Major (was: Critical) > Integrate NamedAggregators into Spark's sink system >

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

2016-11-09 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15650807#comment-15650807 ] Stas Levin commented on BEAM-891: - Perhaps it would be safe to go the other way around, i.e., "only add this

[jira] [Comment Edited] (BEAM-891) Flake in Spark metrics library?

2016-11-09 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15650533#comment-15650533 ] Stas Levin edited comment on BEAM-891 at 11/9/16 12:28 PM: --- We can indeed try

[jira] [Created] (BEAM-816) Aggregators are not properly named when reported to Graphite

2016-10-25 Thread Stas Levin (JIRA)
Stas Levin created BEAM-816: --- Summary: Aggregators are not properly named when reported to Graphite Key: BEAM-816 URL: https://issues.apache.org/jira/browse/BEAM-816 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

2016-11-14 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15663281#comment-15663281 ] Stas Levin commented on BEAM-891: - Sounds good. Btw, why can't we just set

[jira] [Comment Edited] (BEAM-891) Flake in Spark metrics library?

2016-11-14 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15663281#comment-15663281 ] Stas Levin edited comment on BEAM-891 at 11/14/16 9:29 AM: --- Sounds good. Btw, why

[jira] [Closed] (BEAM-1039) Spark context is never actually re-used in tests

2016-11-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin closed BEAM-1039. Resolution: Not A Problem Fix Version/s: Not applicable > Spark context is never actually re-used in

[jira] [Commented] (BEAM-1039) Spark context is never actually re-used in tests

2016-11-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15687906#comment-15687906 ] Stas Levin commented on BEAM-1039: -- For some reason it looked like... Must have been a brain f... My bad,

[jira] [Commented] (BEAM-1050) PipelineResult.State is not set to FAILED in when a streaming job fails

2016-11-27 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15699605#comment-15699605 ] Stas Levin commented on BEAM-1050: -- I see that the code has changed since (the relevant snipped is below),

[jira] [Created] (BEAM-1050) PipelineResult.State is not set to FAILED in when a streaming job fails

2016-11-27 Thread Stas Levin (JIRA)
Stas Levin created BEAM-1050: Summary: PipelineResult.State is not set to FAILED in when a streaming job fails Key: BEAM-1050 URL: https://issues.apache.org/jira/browse/BEAM-1050 Project: Beam

[jira] [Created] (BEAM-1039) Spark context is never actually re-used in tests

2016-11-22 Thread Stas Levin (JIRA)
Stas Levin created BEAM-1039: Summary: Spark context is never actually re-used in tests Key: BEAM-1039 URL: https://issues.apache.org/jira/browse/BEAM-1039 Project: Beam Issue Type: Bug

[jira] [Assigned] (BEAM-1039) Spark context is never actually re-used in tests

2016-11-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-1039: Assignee: Stas Levin (was: Amit Sela) > Spark context is never actually re-used in tests >

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

2016-11-13 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15661096#comment-15661096 ] Stas Levin commented on BEAM-891: - [~amitsela], as far as the name goes, we can try to make it clearer, say

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

2016-11-13 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15661227#comment-15661227 ] Stas Levin commented on BEAM-891: - Cool, turning it off in relevant tests sounds reasonable. we'll need to

[jira] [Comment Edited] (BEAM-891) Flake in Spark metrics library?

2016-11-13 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15661083#comment-15661083 ] Stas Levin edited comment on BEAM-891 at 11/13/16 8:21 AM: --- [~amitsela], as far as

[jira] [Commented] (BEAM-983) runners/spark/translation/streaming/utils/TestPipelineOptions.java missing Apache license header.

2016-11-15 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668207#comment-15668207 ] Stas Levin commented on BEAM-983: - [~jasonkuster] [~amitsela] Sorry about that, can these tests can be added

[jira] [Commented] (BEAM-979) ConcurrentModificationException exception after hours of running

2016-11-15 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667018#comment-15667018 ] Stas Levin commented on BEAM-979: - I believe

[jira] [Created] (BEAM-979) ConcurrentModificationException exception after hours of running

2016-11-15 Thread Stas Levin (JIRA)
Stas Levin created BEAM-979: --- Summary: ConcurrentModificationException exception after hours of running Key: BEAM-979 URL: https://issues.apache.org/jira/browse/BEAM-979 Project: Beam Issue Type:

[jira] [Commented] (BEAM-979) ConcurrentModificationException exception after hours of running

2016-11-15 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15666975#comment-15666975 ] Stas Levin commented on BEAM-979: - I think it may have to do with {StateSpecFunctions#mapSourceFunction}},

[jira] [Comment Edited] (BEAM-979) ConcurrentModificationException exception after hours of running

2016-11-15 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15666975#comment-15666975 ] Stas Levin edited comment on BEAM-979 at 11/15/16 12:24 PM: I think it may have

[jira] [Commented] (BEAM-891) Flake in Spark metrics library?

2016-11-03 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633346#comment-15633346 ] Stas Levin commented on BEAM-891: - Sure. I looked at it a bit this morning, my first impression would be a

[jira] [Assigned] (BEAM-298) Make TestPipeline implement the TestRule interface

2016-12-14 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-298: --- Assignee: Stas Levin > Make TestPipeline implement the TestRule interface >

[jira] [Assigned] (BEAM-85) PAssert needs sanity check that it's used correctly

2016-12-14 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-85: -- Assignee: Stas Levin > PAssert needs sanity check that it's used correctly >

[jira] [Commented] (BEAM-690) Backoff in the DirectRunner Monitor if no work is Available

2016-12-15 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15751670#comment-15751670 ] Stas Levin commented on BEAM-690: - Hi, I'm trying to look into it, correct me if I'm wrong, but won't

[jira] [Updated] (BEAM-1067) apex.examples.WordCountTest.testWordCountExample may be flaky

2016-12-01 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1067: - Description: Seems that {{org.apache.beam.runners.apex.examples.WordCountTest.testWordCountExample}} is

[jira] [Created] (BEAM-1067) apex.examples.WordCountTest.testWordCountExample may be flaky

2016-12-01 Thread Stas Levin (JIRA)
Stas Levin created BEAM-1067: Summary: apex.examples.WordCountTest.testWordCountExample may be flaky Key: BEAM-1067 URL: https://issues.apache.org/jira/browse/BEAM-1067 Project: Beam Issue

[jira] [Commented] (BEAM-1067) apex.examples.WordCountTest.testWordCountExample may be flaky

2016-12-02 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716750#comment-15716750 ] Stas Levin commented on BEAM-1067: -- Not sure, but

[jira] [Commented] (BEAM-1176) Make our test suites use @Rule TestPipeline

2016-12-20 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764919#comment-15764919 ] Stas Levin commented on BEAM-1176: -- Sounds good, enter

[jira] [Created] (BEAM-1186) Migrate the remaining tests to use TestPipeline as a JUnit rule.

2016-12-20 Thread Stas Levin (JIRA)
Stas Levin created BEAM-1186: Summary: Migrate the remaining tests to use TestPipeline as a JUnit rule. Key: BEAM-1186 URL: https://issues.apache.org/jira/browse/BEAM-1186 Project: Beam Issue

[jira] [Comment Edited] (BEAM-1176) Make our test suites use @Rule TestPipeline

2016-12-20 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763964#comment-15763964 ] Stas Levin edited comment on BEAM-1176 at 12/20/16 11:20 AM: - I've started

[jira] [Comment Edited] (BEAM-1176) Make our test suites use @Rule TestPipeline

2016-12-20 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763964#comment-15763964 ] Stas Levin edited comment on BEAM-1176 at 12/20/16 11:18 AM: - I've started

[jira] [Commented] (BEAM-1176) Make our test suites use @Rule TestPipeline

2016-12-20 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763964#comment-15763964 ] Stas Levin commented on BEAM-1176: -- I've started migrating the tests to using the {{TestPipeline}} as a