[jira] [Updated] (BEAM-2464) Create.of(...) does not work as expected when spark-runner is configured to streaming mode

2017-10-03 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2464: - Summary: Create.of(...) does not work as expected when spark-runner is configured to streaming mode (was:

[jira] [Closed] (BEAM-2859) ProcessingTime based timers are not properly fired in case the watermark stays put

2017-09-13 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin closed BEAM-2859. Resolution: Fixed Fix Version/s: 2.2.0 > ProcessingTime based timers are not properly fired in case

[jira] [Updated] (BEAM-2872) AvroIO.TypedWrite#to() method produces a compilation error under JDK 1.7

2017-09-11 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2872: - Summary: AvroIO.TypedWrite#to() method produces a compilation error under JDK 1.7 (was:

[jira] [Comment Edited] (BEAM-2872) AvroIO.TypedWrite#to() method produces a compilation error

2017-09-11 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16161102#comment-16161102 ] Stas Levin edited comment on BEAM-2872 at 9/11/17 11:25 AM: [~reuvenlax] I was

[jira] [Commented] (BEAM-2872) AvroIO.TypedWrite#to() method produces a compilation error

2017-09-11 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16161102#comment-16161102 ] Stas Levin commented on BEAM-2872: -- [~reuvenlax] I was able to make this error go away by setting the

[jira] [Commented] (BEAM-2872) AvroIO.TypedWrite#to() method produces a compilation error

2017-09-11 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16160772#comment-16160772 ] Stas Levin commented on BEAM-2872: -- [~reuvenlax], what JDK are you using? What IDE? You get this error

[jira] [Updated] (BEAM-2872) AvroIO.TypedWrite#to() method produces a compilation error

2017-09-10 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2872: - Description: The following code produces a compilation error when compiled using IntelliJ's {{Rebuild

[jira] [Updated] (BEAM-2872) AvroIO.TypedWrite#to() method produces a compilation error

2017-09-10 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2872: - Description: The following code produces a compilation error when compiled using IntelliJ's {{Rebuild

[jira] [Updated] (BEAM-2872) AvroIO.TypedWrite#to() method produces a compilation error

2017-09-10 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2872: - Summary: AvroIO.TypedWrite#to() method produces a compilation error (was: AvroIO.TypedWrite#to() method

[jira] [Created] (BEAM-2872) AvroIO.TypedWrite#to() method produces a compilation error in IntelliJ (2017.3)

2017-09-10 Thread Stas Levin (JIRA)
Stas Levin created BEAM-2872: Summary: AvroIO.TypedWrite#to() method produces a compilation error in IntelliJ (2017.3) Key: BEAM-2872 URL: https://issues.apache.org/jira/browse/BEAM-2872 Project: Beam

[jira] [Assigned] (BEAM-2872) AvroIO.TypedWrite#to() method produces a compilation error in IntelliJ (2017.3)

2017-09-10 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-2872: Assignee: Reuven Lax (was: Kenneth Knowles) > AvroIO.TypedWrite#to() method produces a compilation

[jira] [Updated] (BEAM-2859) ProcessingTime based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2859: - Summary: ProcessingTime based timers are not properly fired in case the watermark stays put (was:

[jira] [Updated] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2859: - Description: {{AfterProcessingTime}} based timers are not fired when the input watermark does not

[jira] [Updated] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2859: - Description: {{AfterProcessingTime}} based timers are not fired when the input watermark does not

[jira] [Updated] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2859: - Description: {{AfterProcessingTime}} based timers are not fired when the input watermark does not

[jira] [Updated] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2859: - Description: {{AfterProcessingTime}} based timers are not fired when the input watermark does not

[jira] [Updated] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2859: - Description: {{AfterProcessingTime}} based timers are not fired when the input watermark does not

[jira] [Updated] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2859: - Description: {{AfterProcessingTime}} based timers are not fired when the input watermark does not

[jira] [Updated] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2859: - Description: {{AfterProcessingTime}} based timers are not fired when the input watermark does not

[jira] [Updated] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2859: - Affects Version/s: 2.0.0 > Processing time based timers are not properly fired in case the watermark >

[jira] [Created] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
Stas Levin created BEAM-2859: Summary: Processing time based timers are not properly fired in case the watermark stays put Key: BEAM-2859 URL: https://issues.apache.org/jira/browse/BEAM-2859 Project:

[jira] [Assigned] (BEAM-2859) Processing time based timers are not properly fired in case the watermark stays put

2017-09-07 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-2859: Assignee: Stas Levin (was: Amit Sela) > Processing time based timers are not properly fired in

[jira] [Closed] (BEAM-2825) Improve readability of SparkGroupAlsoByWindowViaWindowSet

2017-09-03 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin closed BEAM-2825. Resolution: Fixed Fix Version/s: 2.2.0 > Improve readability of SparkGroupAlsoByWindowViaWindowSet >

[jira] [Closed] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

2017-09-03 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin closed BEAM-2669. Resolution: Fixed > Kryo serialization exception when DStreams containing non-Kryo-serializable > data are

[jira] [Assigned] (BEAM-2825) Improve readability of SparkGroupAlsoByWindowViaWindowSet

2017-08-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-2825: Assignee: Stas Levin (was: Amit Sela) > Improve readability of SparkGroupAlsoByWindowViaWindowSet

[jira] [Closed] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-24 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin closed BEAM-2671. Resolution: Fixed > CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark > runner >

[jira] [Updated] (BEAM-2789) Watermark can become unavailable for executors while it's updated with new values

2017-08-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2789: - Summary: Watermark can become unavailable for executors while it's updated with new values (was:

[jira] [Updated] (BEAM-2789) Watermarks can become unavailable for executors while it's updated with new values

2017-08-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2789: - Summary: Watermarks can become unavailable for executors while it's updated with new values (was:

[jira] [Updated] (BEAM-2789) Watermark can become unavailable for executors while it's updated with new values

2017-08-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2789: - Description: The watermark is updated by the driver like so: {code:java}

[jira] [Created] (BEAM-2789) Watermark can become unavailable for executors while it's updated with new values

2017-08-22 Thread Stas Levin (JIRA)
Stas Levin created BEAM-2789: Summary: Watermark can become unavailable for executors while it's updated with new values Key: BEAM-2789 URL: https://issues.apache.org/jira/browse/BEAM-2789 Project: Beam

[jira] [Assigned] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-2671: Assignee: Stas Levin (was: Jean-Baptiste Onofré) > CreateStreamTest.testFirstElementLate

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-16 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129014#comment-16129014 ] Stas Levin edited comment on BEAM-2671 at 8/16/17 4:13 PM: --- I believe I've been

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-16 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129014#comment-16129014 ] Stas Levin commented on BEAM-2671: -- I believe I've been able to pinpoint the problem and am working

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-13 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124897#comment-16124897 ] Stas Levin commented on BEAM-2671: -- [~kenn], could you elaborate on when there might be a race, and what

[jira] [Assigned] (BEAM-2670) ParDoTest.testPipelineOptionsParameter* validatesRunner tests fail on Spark runner

2017-07-25 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-2670: Assignee: Stas Levin (was: Amit Sela) > ParDoTest.testPipelineOptionsParameter* validatesRunner

[jira] [Assigned] (BEAM-1712) TestPipeline.run should perhaps waitUntilFinish

2017-06-18 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-1712: Assignee: (was: Stas Levin) > TestPipeline.run should perhaps waitUntilFinish >

[jira] [Assigned] (BEAM-2464) Create.of(...) does not work as expected when runner is configured to streaming mode

2017-06-18 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-2464: Assignee: Stas Levin (was: Amit Sela) > Create.of(...) does not work as expected when runner is

[jira] [Created] (BEAM-2464) Create.of(...) is not properly handled when runner is configured to streaming mode

2017-06-18 Thread Stas Levin (JIRA)
Stas Levin created BEAM-2464: Summary: Create.of(...) is not properly handled when runner is configured to streaming mode Key: BEAM-2464 URL: https://issues.apache.org/jira/browse/BEAM-2464 Project: Beam

[jira] [Updated] (BEAM-2464) Create.of(...) does not work as expected when runner is configured to streaming mode

2017-06-18 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2464: - Summary: Create.of(...) does not work as expected when runner is configured to streaming mode (was:

[jira] [Comment Edited] (BEAM-2095) The hasNext method of the iterator returned by SourceRDD#compute is not idempotent

2017-05-11 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006299#comment-16006299 ] Stas Levin edited comment on BEAM-2095 at 5/11/17 12:04 PM: Hi Arvid, Sorry I

[jira] [Comment Edited] (BEAM-2095) The hasNext method of the iterator returned by SourceRDD#compute is not idempotent

2017-05-11 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006299#comment-16006299 ] Stas Levin edited comment on BEAM-2095 at 5/11/17 12:01 PM: Hi Arvid, Sorry I

[jira] [Commented] (BEAM-2095) The hasNext method of the iterator returned by SourceRDD#compute is not idempotent

2017-05-11 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006299#comment-16006299 ] Stas Levin commented on BEAM-2095: -- Sorry I did not provide any context here. I was hoping you'd be

[jira] [Commented] (BEAM-2095) The hasNext method of the iterator returned by SourceRDD#compute is not idempotent

2017-05-09 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002206#comment-16002206 ] Stas Levin commented on BEAM-2095: -- [~arvid heise] this fix is currently in the master branch and will

[jira] [Resolved] (BEAM-2095) The hasNext method of the iterator returned by SourceRDD#compute is not idempotent

2017-05-09 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin resolved BEAM-2095. -- Resolution: Fixed Fix Version/s: 2.1.0 > The hasNext method of the iterator returned by

[jira] [Updated] (BEAM-2095) The hasNext method of the iterator returned by SourceRDD#compute is not idempotent

2017-05-03 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-2095: - Summary: The hasNext method of the iterator returned by SourceRDD#compute is not idempotent (was:

[jira] [Resolved] (BEAM-2073) Change SourceDStream.rateControlledMaxRecords() to better reflect its intention

2017-05-01 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin resolved BEAM-2073. -- Resolution: Fixed Fix Version/s: First stable release > Change

[jira] [Resolved] (BEAM-2074) SourceDStream's rate control mechanism may not work

2017-05-01 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin resolved BEAM-2074. -- Resolution: Fixed Fix Version/s: First stable release > SourceDStream's rate control mechanism

[jira] [Resolved] (BEAM-2072) MicrobatchSource.reader stops reading after reaching maxNumRecords for the first time

2017-05-01 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin resolved BEAM-2072. -- Resolution: Fixed Fix Version/s: First stable release > MicrobatchSource.reader stops reading

[jira] [Assigned] (BEAM-2095) SourceRDD hasNext not idempotent

2017-04-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-2095: Assignee: Stas Levin (was: Amit Sela) > SourceRDD hasNext not idempotent >

[jira] [Comment Edited] (BEAM-1676) SdkCoreApiSurfaceTest Failed When Directory Contains Space

2017-04-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990193#comment-15990193 ] Stas Levin edited comment on BEAM-1676 at 4/30/17 11:32 AM: We can internalize

[jira] [Comment Edited] (BEAM-1676) SdkCoreApiSurfaceTest Failed When Directory Contains Space

2017-04-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990193#comment-15990193 ] Stas Levin edited comment on BEAM-1676 at 4/30/17 11:31 AM: We can internalize

[jira] [Commented] (BEAM-1676) SdkCoreApiSurfaceTest Failed When Directory Contains Space

2017-04-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15990193#comment-15990193 ] Stas Levin commented on BEAM-1676: -- We can internalise (euphemism for copy-paste) the fixed {{ClassPath}}

[jira] [Commented] (BEAM-1777) If PipelineEnforcement throws an exception after Pipeline.run() fails, it overwrites the original failure

2017-04-27 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986425#comment-15986425 ] Stas Levin commented on BEAM-1777: -- [~tgroh] do you believe we can mark this one as resolved? > If

[jira] [Created] (BEAM-2092) MicrobatchSource can be relieved of some of its methods since it's never used as an actual BoundedSource

2017-04-27 Thread Stas Levin (JIRA)
Stas Levin created BEAM-2092: Summary: MicrobatchSource can be relieved of some of its methods since it's never used as an actual BoundedSource Key: BEAM-2092 URL: https://issues.apache.org/jira/browse/BEAM-2092

[jira] [Created] (BEAM-2074) SourceDStream's rate control mechanism may not work

2017-04-25 Thread Stas Levin (JIRA)
Stas Levin created BEAM-2074: Summary: SourceDStream's rate control mechanism may not work Key: BEAM-2074 URL: https://issues.apache.org/jira/browse/BEAM-2074 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-2073) Change SourceDStream.rateControlledMaxRecords() to better reflect its intention

2017-04-25 Thread Stas Levin (JIRA)
Stas Levin created BEAM-2073: Summary: Change SourceDStream.rateControlledMaxRecords() to better reflect its intention Key: BEAM-2073 URL: https://issues.apache.org/jira/browse/BEAM-2073 Project: Beam

[jira] [Created] (BEAM-2072) MicrobatchSource.reader stops reading after reaching maxNumRecords for the first time

2017-04-25 Thread Stas Levin (JIRA)
Stas Levin created BEAM-2072: Summary: MicrobatchSource.reader stops reading after reaching maxNumRecords for the first time Key: BEAM-2072 URL: https://issues.apache.org/jira/browse/BEAM-2072 Project:

[jira] [Assigned] (BEAM-1295) Annotations are not properly rendered in JavaDoc when using {@code}

2017-04-12 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-1295: Assignee: (was: Stas Levin) > Annotations are not properly rendered in JavaDoc when using

[jira] [Comment Edited] (BEAM-1777) If PipelineEnforcement throws an exception after Pipeline.run() fails, it overwrites the original failure

2017-04-05 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956479#comment-15956479 ] Stas Levin edited comment on BEAM-1777 at 4/5/17 8:01 AM: -- [~tgroh] since I was

[jira] [Commented] (BEAM-1777) If PipelineEnforcement throws an exception after Pipeline.run() fails, it overwrites the original failure

2017-04-05 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956479#comment-15956479 ] Stas Levin commented on BEAM-1777: -- [~tgroh] Since I was unable to come up with a proper test for this,

[jira] [Comment Edited] (BEAM-1777) If PipelineEnforcement throws an exception after Pipeline.run() fails, it overwrites the original failure

2017-04-05 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956479#comment-15956479 ] Stas Levin edited comment on BEAM-1777 at 4/5/17 8:00 AM: -- [~tgroh] since I was

[jira] [Commented] (BEAM-1048) Spark Runner streaming batch duration does not include duration of reading from source

2017-04-02 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15952616#comment-15952616 ] Stas Levin commented on BEAM-1048: -- While batch duration is still (and probably won't be) reported as part

[jira] [Resolved] (BEAM-1048) Spark Runner streaming batch duration does not include duration of reading from source

2017-04-02 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin resolved BEAM-1048. -- Resolution: Workaround > Spark Runner streaming batch duration does not include duration of reading >

[jira] [Updated] (BEAM-1048) Spark Runner streaming batch duration does not include duration of reading from source

2017-04-02 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1048: - Fix Version/s: First stable release > Spark Runner streaming batch duration does not include duration of

[jira] [Updated] (BEAM-1839) Optimize StatelessJavaSerializer

2017-03-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1839: - Summary: Optimize StatelessJavaSerializer (was: Opitimize StatelessJavaSerializer) > Optimize

[jira] [Updated] (BEAM-1839) Opitimize StatelessJavaSerializer

2017-03-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1839: - Description: Since Kryo uses reflection to sequentially look for constructor signatures, we can prevent

[jira] [Resolved] (BEAM-1713) SparkRuntimeContext instances are leaking via StateSpecFunctions#mapSourceFunction

2017-03-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin resolved BEAM-1713. -- Resolution: Fixed Fix Version/s: First stable release > SparkRuntimeContext instances are leaking

[jira] [Resolved] (BEAM-1839) Opitimize StatelessJavaSerializer

2017-03-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin resolved BEAM-1839. -- Resolution: Fixed Fix Version/s: First stable release > Opitimize StatelessJavaSerializer >

[jira] [Created] (BEAM-1839) Opitimize StatelessJavaSerializer

2017-03-30 Thread Stas Levin (JIRA)
Stas Levin created BEAM-1839: Summary: Opitimize StatelessJavaSerializer Key: BEAM-1839 URL: https://issues.apache.org/jira/browse/BEAM-1839 Project: Beam Issue Type: Improvement

[jira] [Resolved] (BEAM-1828) GlobalWatermarkHolder uses unpersist instead of destory

2017-03-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin resolved BEAM-1828. -- Resolution: Fixed Fix Version/s: First stable release > GlobalWatermarkHolder uses unpersist

[jira] [Commented] (BEAM-1676) SdkCoreApiSurfaceTest Failed When Directory Contains Space

2017-03-30 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948913#comment-15948913 ] Stas Levin commented on BEAM-1676: -- This has recently been fixed in {{Guava}}'s

[jira] [Assigned] (BEAM-1828) GlobalWatermarkHolder uses unpersist instead of destory

2017-03-29 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-1828: Assignee: Stas Levin (was: Amit Sela) > GlobalWatermarkHolder uses unpersist instead of destory >

[jira] [Created] (BEAM-1828) GlobalWatermarkHolder uses unpersist instead of destory

2017-03-29 Thread Stas Levin (JIRA)
Stas Levin created BEAM-1828: Summary: GlobalWatermarkHolder uses unpersist instead of destory Key: BEAM-1828 URL: https://issues.apache.org/jira/browse/BEAM-1828 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-1499) Update testing documentation at beam-site

2017-03-28 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944993#comment-15944993 ] Stas Levin commented on BEAM-1499: -- [~jasonkuster], we can add some info following the change of

[jira] [Updated] (BEAM-1712) TestPipeline.run should perhaps waitUntilFinish

2017-03-28 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1712: - Priority: Minor (was: Major) > TestPipeline.run should perhaps waitUntilFinish >

[jira] [Comment Edited] (BEAM-1777) If PipelineEnforcement throws an exception after Pipeline.run() fails, it overwrites the original failure

2017-03-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936232#comment-15936232 ] Stas Levin edited comment on BEAM-1777 at 3/22/17 12:47 PM: I wonder if this

[jira] [Commented] (BEAM-1777) If PipelineEnforcement throws an exception after Pipeline.run() fails, it overwrites the original failure

2017-03-22 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936232#comment-15936232 ] Stas Levin commented on BEAM-1777: -- I wonder if this can be resolved by removing the {{finally}} around

[jira] [Commented] (BEAM-1676) SdkCoreApiSurfaceTest Failed When Directory Contains Space

2017-03-19 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931625#comment-15931625 ] Stas Levin commented on BEAM-1676: -- Good catch [~markflyhigh], looks like it has to do with [this issue in

[jira] [Updated] (BEAM-1713) SparkRuntimeContext instances are leaking via StateSpecFunctions#mapSourceFunction

2017-03-14 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1713: - Description: When creating an anonymous class like so: {code:java}new Iterable() {...}{code} it is

[jira] [Assigned] (BEAM-1713) {{SparkRuntimeContext}} instances are leaking via {{StateSpecFunctions#mapSourceFunction}}

2017-03-14 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin reassigned BEAM-1713: Assignee: Stas Levin (was: Amit Sela) > {{SparkRuntimeContext}} instances are leaking via >

[jira] [Updated] (BEAM-1713) SparkRuntimeContext instances are leaking via StateSpecFunctions#mapSourceFunction

2017-03-14 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1713: - Summary: SparkRuntimeContext instances are leaking via StateSpecFunctions#mapSourceFunction (was:

[jira] [Created] (BEAM-1713) {{SparkRuntimeContext}} instances are leaking via {{StateSpecFunctions#mapSourceFunction}}

2017-03-14 Thread Stas Levin (JIRA)
Stas Levin created BEAM-1713: Summary: {{SparkRuntimeContext}} instances are leaking via {{StateSpecFunctions#mapSourceFunction}} Key: BEAM-1713 URL: https://issues.apache.org/jira/browse/BEAM-1713

[jira] [Comment Edited] (BEAM-1712) TestPipeline.run doesn't actually waitUntilFinish

2017-03-14 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923650#comment-15923650 ] Stas Levin edited comment on BEAM-1712 at 3/14/17 6:20 AM: --- [A related

[jira] [Commented] (BEAM-1712) TestPipeline.run doesn't actually waitUntilFinish

2017-03-14 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923650#comment-15923650 ] Stas Levin commented on BEAM-1712: -- [A related

[jira] [Updated] (BEAM-1557) Separate TestPipeline into 2 versions: one for users, and one for sdk/runner developers

2017-03-01 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1557: - Description: At the moment, {{TestPipeline}} enforces some usage patterns, that may be too strict for non

[jira] [Commented] (BEAM-1543) TestPipeline: @Rule error message confusing/javadoc doesn't explain proper usage

2017-02-26 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884847#comment-15884847 ] Stas Levin commented on BEAM-1543: -- [~sisk], we did some [brush ups|

[jira] [Updated] (BEAM-1557) Separate TestPipeline into 2 versions: one for users, and one for sdk/runner developers

2017-02-26 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1557: - Description: At the moment, {{TestPipeline}} has enforces some usage patterns, that may be too strict for

[jira] [Updated] (BEAM-1557) Separate TestPipeline into 2 versions: one for users, and one for sdk/runner developers

2017-02-26 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-1557: - Description: At the moment, {{TestPipeline}} has enforces some usage patterns, that may be too strict for

[jira] [Created] (BEAM-1557) Separate TestPipeline into 2 versions: one for users, and one for sdk/runner developers

2017-02-26 Thread Stas Levin (JIRA)
Stas Levin created BEAM-1557: Summary: Separate TestPipeline into 2 versions: one for users, and one for sdk/runner developers Key: BEAM-1557 URL: https://issues.apache.org/jira/browse/BEAM-1557 Project:

[jira] [Comment Edited] (BEAM-1499) Update testing documentation at beam-site

2017-02-24 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882918#comment-15882918 ] Stas Levin edited comment on BEAM-1499 at 2/24/17 3:45 PM: --- [~jasonkuster], I've

[jira] [Commented] (BEAM-1499) Update testing documentation at beam-site

2017-02-24 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882918#comment-15882918 ] Stas Levin commented on BEAM-1499: -- [~jasonkuster], I've added two paragraphs under the contribute/testing

[jira] [Comment Edited] (BEAM-1543) TestPipeline: @Rule error message confusing/javadoc doesn't explain proper usage

2017-02-23 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880974#comment-15880974 ] Stas Levin edited comment on BEAM-1543 at 2/23/17 6:26 PM: --- [~kenn] definitely,

[jira] [Commented] (BEAM-1543) TestPipeline: @Rule error message confusing/javadoc doesn't explain proper usage

2017-02-23 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880974#comment-15880974 ] Stas Levin commented on BEAM-1543: -- Definitely, I'll take care of it. Did this PR assist in resolving the

[jira] [Comment Edited] (BEAM-1399) Code coverage numbers are not accurate

2017-02-20 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873590#comment-15873590 ] Stas Levin edited comment on BEAM-1399 at 2/20/17 8:04 AM: --- Thanks for the tip

[jira] [Comment Edited] (BEAM-1399) Code coverage numbers are not accurate

2017-02-20 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873590#comment-15873590 ] Stas Levin edited comment on BEAM-1399 at 2/20/17 8:04 AM: --- Thanks for the tip

[jira] [Commented] (BEAM-1499) Update testing documentation at beam-site

2017-02-18 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873270#comment-15873270 ] Stas Levin commented on BEAM-1499: -- [~davor], sure thing. Thanks for helping out! Looking forward to get

[jira] [Commented] (BEAM-1499) Update testing documentation at beam-site

2017-02-16 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870539#comment-15870539 ] Stas Levin commented on BEAM-1499: -- I'd be glad to lend a hand as well, just say the word :) > Update

[jira] [Created] (BEAM-1499) Update testing documentation at beam-site

2017-02-16 Thread Stas Levin (JIRA)
Stas Levin created BEAM-1499: Summary: Update testing documentation at beam-site Key: BEAM-1499 URL: https://issues.apache.org/jira/browse/BEAM-1499 Project: Beam Issue Type: Improvement

[jira] [Updated] (BEAM-883) Make ApiSurfaceTest fail if something whitelisted is _not_ exposed

2017-02-16 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-883: Fix Version/s: (was: Not applicable) 0.6.0 > Make ApiSurfaceTest fail if something

[jira] [Updated] (BEAM-882) Make ApiSurfaceTest detect the java package/module under test

2017-02-16 Thread Stas Levin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stas Levin updated BEAM-882: Fix Version/s: (was: Not applicable) 0.6.0 > Make ApiSurfaceTest detect the java

  1   2   >