[jira] [Created] (BEAM-1372) OutputTimeFn and Accumulating Mode is Confusing

2017-02-01 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1372: - Summary: OutputTimeFn and Accumulating Mode is Confusing Key: BEAM-1372 URL: https://issues.apache.org/jira/browse/BEAM-1372 Project: Beam Issue Type: Bug

[jira] [Closed] (BEAM-1370) AfterWatermarkEarlyAndLate does not invoke the onMerge of the early trigger

2017-02-02 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh closed BEAM-1370. - Resolution: Fixed Fix Version/s: 0.5.0 fixed in {{0a901d9489afec98c4b0d6db02e4837030e2a404}} >

[jira] [Created] (BEAM-1370) AfterWatermarkEarlyAndLate does not invoke the onMerge of the early trigger

2017-02-01 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1370: - Summary: AfterWatermarkEarlyAndLate does not invoke the onMerge of the early trigger Key: BEAM-1370 URL: https://issues.apache.org/jira/browse/BEAM-1370 Project: Beam

[jira] [Resolved] (BEAM-1409) The DirectRunner should forbid elements outside of representable time.

2017-02-07 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1409. --- Resolution: Fixed Assignee: Thomas Groh (was: Davor Bonaci) Fix Version/s: 0.6.0 >

[jira] [Commented] (BEAM-1409) The DirectRunner should forbid elements outside of representable time.

2017-02-07 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856343#comment-15856343 ] Thomas Groh commented on BEAM-1409: --- The DirectRunner enforces that element timestamps and timer

[jira] [Updated] (BEAM-1409) The DirectRunner forbids elements outside of representable time.

2017-02-07 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-1409: -- Summary: The DirectRunner forbids elements outside of representable time. (was: The DirectRunner should

[jira] [Created] (BEAM-1409) The DirectRunner should forbid elements outside of representable time.

2017-02-06 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1409: - Summary: The DirectRunner should forbid elements outside of representable time. Key: BEAM-1409 URL: https://issues.apache.org/jira/browse/BEAM-1409 Project: Beam

[jira] [Commented] (BEAM-1316) DoFn#startBundle and #finishBundle should not be able to output

2017-01-25 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838761#comment-15838761 ] Thomas Groh commented on BEAM-1316: --- Forbidding output from startBundle and finishBundle brings the

[jira] [Resolved] (BEAM-1311) Flink Translations are not properly 1-to-1 with the PValues, PInputs, or POutputs

2017-01-25 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1311. --- Resolution: Fixed Fix Version/s: Not applicable > Flink Translations are not properly 1-to-1

[jira] [Created] (BEAM-1316) DoFn#startBundle and #finishBundle should not be able to output

2017-01-25 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1316: - Summary: DoFn#startBundle and #finishBundle should not be able to output Key: BEAM-1316 URL: https://issues.apache.org/jira/browse/BEAM-1316 Project: Beam Issue

[jira] [Created] (BEAM-1311) Flink Translations are not properly 1-to-1 with the PValues, PInputs, or POutputs

2017-01-24 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1311: - Summary: Flink Translations are not properly 1-to-1 with the PValues, PInputs, or POutputs Key: BEAM-1311 URL: https://issues.apache.org/jira/browse/BEAM-1311 Project:

[jira] [Commented] (BEAM-1311) Flink Translations are not properly 1-to-1 with the PValues, PInputs, or POutputs

2017-01-24 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837060#comment-15837060 ] Thomas Groh commented on BEAM-1311: --- Basically I munged a call to {{getTag()}} and a call to

[jira] [Resolved] (BEAM-1523) Remove PValue.getProducingTransformInternal

2017-02-21 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1523. --- Resolution: Fixed Fix Version/s: 0.6.0 > Remove PValue.getProducingTransformInternal >

[jira] [Assigned] (BEAM-1526) Flakes in Spark runner WatermarkTest.testInDoFn

2017-02-21 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-1526: - Assignee: Thomas Groh (was: Amit Sela) > Flakes in Spark runner WatermarkTest.testInDoFn >

[jira] [Created] (BEAM-1508) PInput, POutput#expand should not be ordered

2017-02-17 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1508: - Summary: PInput, POutput#expand should not be ordered Key: BEAM-1508 URL: https://issues.apache.org/jira/browse/BEAM-1508 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-1548) Window should not reassign windows if no WindowFn is specified

2017-02-24 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1548: - Summary: Window should not reassign windows if no WindowFn is specified Key: BEAM-1548 URL: https://issues.apache.org/jira/browse/BEAM-1548 Project: Beam Issue

[jira] [Created] (BEAM-1551) Allow PAssert#that to take a message

2017-02-24 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1551: - Summary: Allow PAssert#that to take a message Key: BEAM-1551 URL: https://issues.apache.org/jira/browse/BEAM-1551 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-1553) OutputTimeFn maybe should be allowed to move elements back in time

2017-02-24 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1553: - Summary: OutputTimeFn maybe should be allowed to move elements back in time Key: BEAM-1553 URL: https://issues.apache.org/jira/browse/BEAM-1553 Project: Beam

[jira] [Created] (BEAM-1523) Remove PValue.getProducingTransformInternal

2017-02-21 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1523: - Summary: Remove PValue.getProducingTransformInternal Key: BEAM-1523 URL: https://issues.apache.org/jira/browse/BEAM-1523 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-1461) duplication with StartBundle and prepareForProcessing in DoFn

2017-02-13 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864666#comment-15864666 ] Thomas Groh commented on BEAM-1461: --- Sure; {{prepareForProcessing}} exists purely to support Aggregators

[jira] [Created] (BEAM-1455) Python hangs after running unit tests

2017-02-09 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1455: - Summary: Python hangs after running unit tests Key: BEAM-1455 URL: https://issues.apache.org/jira/browse/BEAM-1455 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-1487) BufferingStreamObserverTest, BeamFnLoggingClientTest repeatedly times out in precommit

2017-02-14 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1487: - Summary: BufferingStreamObserverTest, BeamFnLoggingClientTest repeatedly times out in precommit Key: BEAM-1487 URL: https://issues.apache.org/jira/browse/BEAM-1487

[jira] [Created] (BEAM-1489) The DirectRunner should not discard existing PaneInfo during Window Assignment

2017-02-14 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1489: - Summary: The DirectRunner should not discard existing PaneInfo during Window Assignment Key: BEAM-1489 URL: https://issues.apache.org/jira/browse/BEAM-1489 Project: Beam

[jira] [Updated] (BEAM-646) Get runners out of the apply()

2017-02-14 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-646: - Labels: backwards-incompatible (was: ) > Get runners out of the apply() > -- >

[jira] [Updated] (BEAM-1489) The DirectRunner does not discard existing PaneInfo during Window Assignment

2017-02-14 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-1489: -- Summary: The DirectRunner does not discard existing PaneInfo during Window Assignment (was: The

[jira] [Resolved] (BEAM-1489) The DirectRunner should not discard existing PaneInfo during Window Assignment

2017-02-14 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1489. --- Resolution: Fixed Fix Version/s: 0.6.0 > The DirectRunner should not discard existing PaneInfo

[jira] [Created] (BEAM-1446) Create should take a TypeDescriptor as an alternative to explicitly specifying the Coder

2017-02-09 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1446: - Summary: Create should take a TypeDescriptor as an alternative to explicitly specifying the Coder Key: BEAM-1446 URL: https://issues.apache.org/jira/browse/BEAM-1446

[jira] [Commented] (BEAM-1495) Move ParDo.Bound as ParDo.BoundMulti into the core runners construction module

2017-02-15 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868989#comment-15868989 ] Thomas Groh commented on BEAM-1495: --- It is worth instead considering implementing ParDo.Bound purely in

[jira] [Created] (BEAM-1493) runners/core-java should be a pre-execution and an execution-time module

2017-02-15 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1493: - Summary: runners/core-java should be a pre-execution and an execution-time module Key: BEAM-1493 URL: https://issues.apache.org/jira/browse/BEAM-1493 Project: Beam

[jira] [Assigned] (BEAM-865) Support set/delete of timer by ID in DirectRunner

2017-02-15 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-865: Assignee: Kenneth Knowles (was: Thomas Groh) > Support set/delete of timer by ID in DirectRunner >

[jira] [Resolved] (BEAM-1300) The DirectRunner resumes from unfinalized checkpoints

2017-02-15 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1300. --- Resolution: Fixed Fix Version/s: 0.5.0 > The DirectRunner resumes from unfinalized checkpoints

[jira] [Created] (BEAM-1495) Move ParDo.Bound as ParDo.BoundMulti into the core runners construction module

2017-02-15 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1495: - Summary: Move ParDo.Bound as ParDo.BoundMulti into the core runners construction module Key: BEAM-1495 URL: https://issues.apache.org/jira/browse/BEAM-1495 Project: Beam

[jira] [Assigned] (BEAM-440) Create.values() returns a type-unsafe Coder

2017-02-09 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-440: Assignee: Thomas Groh (was: Jason White) > Create.values() returns a type-unsafe Coder >

[jira] [Resolved] (BEAM-440) Create.values() returns a type-unsafe Coder

2017-02-09 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-440. -- Resolution: Fixed Fix Version/s: 0.6.0 > Create.values() returns a type-unsafe Coder >

[jira] [Created] (BEAM-1279) Composite nodes should never be able to produce primitive output

2017-01-18 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1279: - Summary: Composite nodes should never be able to produce primitive output Key: BEAM-1279 URL: https://issues.apache.org/jira/browse/BEAM-1279 Project: Beam Issue

[jira] [Resolved] (BEAM-1548) Window should not reassign windows if no WindowFn is specified

2017-03-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1548. --- Resolution: Fixed Fix Version/s: 0.6.0 > Window should not reassign windows if no WindowFn is

[jira] [Assigned] (BEAM-1173) TestSparkRunner should use a PipelineVisitor to determine expected assertions.

2017-03-01 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-1173: - Assignee: Thomas Groh > TestSparkRunner should use a PipelineVisitor to determine expected

[jira] [Resolved] (BEAM-1446) Create should take a TypeDescriptor as an alternative to explicitly specifying the Coder

2017-02-27 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1446. --- Resolution: Fixed Fix Version/s: 0.6.0 > Create should take a TypeDescriptor as an alternative

[jira] [Created] (BEAM-1571) Flatten on a single input PCollection should have a test associated with it

2017-02-27 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1571: - Summary: Flatten on a single input PCollection should have a test associated with it Key: BEAM-1571 URL: https://issues.apache.org/jira/browse/BEAM-1571 Project: Beam

[jira] [Commented] (BEAM-1249) Flatten with heterongeneous coders does not have a RunnableOnService test

2017-01-06 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805287#comment-15805287 ] Thomas Groh commented on BEAM-1249: --- Resolution for runners is currently up to the runner. There's no

[jira] [Created] (BEAM-1210) PipelineRunners should receive a copy of PipelineOptions rather than the original Options

2016-12-22 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1210: - Summary: PipelineRunners should receive a copy of PipelineOptions rather than the original Options Key: BEAM-1210 URL: https://issues.apache.org/jira/browse/BEAM-1210

[jira] [Commented] (BEAM-1205) Auto set "enableAbandonedNodeEnforcement" in TestPipeline

2016-12-22 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770623#comment-15770623 ] Thomas Groh commented on BEAM-1205: --- If a user fails to include the {{DirectRunner}} on the classpath,

[jira] [Commented] (BEAM-1210) PipelineRunners should receive a copy of PipelineOptions rather than the original Options

2016-12-22 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770714#comment-15770714 ] Thomas Groh commented on BEAM-1210: --- It is significantly easier to understand the scope of options and

[jira] [Commented] (BEAM-1205) Auto set "enableAbandonedNodeEnforcement" in TestPipeline

2016-12-22 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770573#comment-15770573 ] Thomas Groh commented on BEAM-1205: --- I really like this idea, but there are a couple of things we should

[jira] [Created] (BEAM-1796) Apex RunnableOnService fails on testShiftTimestampInvalid

2017-03-23 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1796: - Summary: Apex RunnableOnService fails on testShiftTimestampInvalid Key: BEAM-1796 URL: https://issues.apache.org/jira/browse/BEAM-1796 Project: Beam Issue Type:

[jira] [Resolved] (BEAM-1447) Autodetect streaming/not streaming in DataflowRunner

2017-03-24 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1447. --- Resolution: Fixed Fix Version/s: First stable release > Autodetect streaming/not streaming in

[jira] [Updated] (BEAM-1818) Expose side-channel inputs in PTransform

2017-03-27 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-1818: -- Description: This enables all PTransforms to consume inputs without using those inputs provided in the

[jira] [Created] (BEAM-1818) Expose side-channel inputs in PTransform

2017-03-27 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1818: - Summary: Expose side-channel inputs in PTransform Key: BEAM-1818 URL: https://issues.apache.org/jira/browse/BEAM-1818 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-1819) Key should be available in @OnTimer methods

2017-03-27 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1819: - Summary: Key should be available in @OnTimer methods Key: BEAM-1819 URL: https://issues.apache.org/jira/browse/BEAM-1819 Project: Beam Issue Type: Bug

[jira] [Resolved] (BEAM-1182) Direct runner should enforce encodability of unbounded source checkpoints

2017-03-24 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1182. --- Resolution: Fixed Fix Version/s: First stable release > Direct runner should enforce

[jira] [Commented] (BEAM-1579) Runners should verify that PT overrides converged

2017-03-30 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15949415#comment-15949415 ] Thomas Groh commented on BEAM-1579: --- The Pipeline apis support this, and the DirectRunner uses that API.

[jira] [Created] (BEAM-1842) Stop matching composite PCollectionView PTransforms

2017-03-30 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1842: - Summary: Stop matching composite PCollectionView PTransforms Key: BEAM-1842 URL: https://issues.apache.org/jira/browse/BEAM-1842 Project: Beam Issue Type: Bug

[jira] [Resolved] (BEAM-1571) Flatten on a single input PCollection should have a test associated with it

2017-03-22 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1571. --- Resolution: Fixed Fix Version/s: First stable release > Flatten on a single input PCollection

[jira] [Resolved] (BEAM-596) Support cancel() and waitUntilFinish() in DirectRunner

2017-03-22 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-596. -- Resolution: Fixed Fix Version/s: First stable release > Support cancel() and waitUntilFinish() in

[jira] [Assigned] (BEAM-1493) runners/core-java should be a pre-execution and an execution-time module

2017-03-22 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-1493: - Assignee: (was: Thomas Groh) I'm not actively working on the migration of anything that is

[jira] [Resolved] (BEAM-1658) DirectRunner should sometimes encode/decode checkpoints

2017-03-22 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1658. --- Resolution: Duplicate Fix Version/s: Not applicable > DirectRunner should sometimes

[jira] [Resolved] (BEAM-1718) Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew() causes Overflow/Underflow

2017-03-22 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1718. --- Resolution: Fixed Fix Version/s: First stable release > Returning

[jira] [Updated] (BEAM-576) ReduceFnRunner should GC based on current WM, not on a fired Timer

2017-03-23 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-576: - Description: ReduceFnRunner currently uses the time of a timer to determine if it is the garbage collection

[jira] [Commented] (BEAM-576) ReduceFnRunner should GC based on current WM, not on a fired Timer

2017-03-23 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938618#comment-15938618 ] Thomas Groh commented on BEAM-576: -- Yes. Additionally, if we don't set a GC timer we won't promptly expire

[jira] [Commented] (BEAM-1835) NPE in DirectRunner PubsubReader.ackBatch

2017-03-29 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948285#comment-15948285 ] Thomas Groh commented on BEAM-1835: --- This is actually a bug in PubsubUnboundedSource or PubsubCheckpoint.

[jira] [Resolved] (BEAM-1743) View.AsSingleton should be implemented in terms of a Global Combine, not the reverse

2017-03-28 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1743. --- Resolution: Fixed > View.AsSingleton should be implemented in terms of a Global Combine, not the >

[jira] [Assigned] (BEAM-1578) Runners should put PT overrides into a list rather than map

2017-03-31 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-1578: - Assignee: Thomas Groh > Runners should put PT overrides into a list rather than map >

[jira] [Resolved] (BEAM-1709) Implement Single-output ParDo as Multi-output ParDo

2017-03-17 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1709. --- Resolution: Fixed > Implement Single-output ParDo as Multi-output ParDo >

[jira] [Resolved] (BEAM-1721) Reshuffle can shift elements in time

2017-03-17 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1721. --- Resolution: Fixed Fix Version/s: First stable release > Reshuffle can shift elements in time >

[jira] [Commented] (BEAM-1738) DataflowRunner should override Reshuffle transform

2017-03-17 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930236#comment-15930236 ] Thomas Groh commented on BEAM-1738: --- Dataflow internally does no buffering and outputs elements with

[jira] [Created] (BEAM-1743) View.AsSingleton should be implemented in terms of a Global Combine, not the reverse

2017-03-17 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1743: - Summary: View.AsSingleton should be implemented in terms of a Global Combine, not the reverse Key: BEAM-1743 URL: https://issues.apache.org/jira/browse/BEAM-1743 Project:

[jira] [Created] (BEAM-1742) UnboundedSource CheckpointMark should have more precise documentation

2017-03-17 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1742: - Summary: UnboundedSource CheckpointMark should have more precise documentation Key: BEAM-1742 URL: https://issues.apache.org/jira/browse/BEAM-1742 Project: Beam

[jira] [Resolved] (BEAM-1653) Error when using PubsubIO with the DirectRunner

2017-03-17 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1653. --- Resolution: Fixed Fix Version/s: First stable release > Error when using PubsubIO with the

[jira] [Commented] (BEAM-1653) Error when using PubsubIO with the DirectRunner

2017-03-17 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930846#comment-15930846 ] Thomas Groh commented on BEAM-1653: --- This should be fixed in HEAD. There are future changes to

[jira] [Resolved] (BEAM-1657) DirectRunner should not call close twice in UnboundedSourceExecutorFactory

2017-03-14 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1657. --- Resolution: Fixed Fix Version/s: First stable release > DirectRunner should not call close

[jira] [Resolved] (BEAM-1656) DirectRunner should not call finalize twice in UnboundedSourceExecutorFactory

2017-03-14 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1656. --- Resolution: Fixed Fix Version/s: First stable release > DirectRunner should not call finalize

[jira] [Created] (BEAM-1732) Window.Assign does not properly populate DisplayData of the enclosing Window transform

2017-03-15 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1732: - Summary: Window.Assign does not properly populate DisplayData of the enclosing Window transform Key: BEAM-1732 URL: https://issues.apache.org/jira/browse/BEAM-1732

[jira] [Created] (BEAM-1721) Reshuffle can shift elements in time

2017-03-14 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1721: - Summary: Reshuffle can shift elements in time Key: BEAM-1721 URL: https://issues.apache.org/jira/browse/BEAM-1721 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-1718) Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew() causes Overflow/Underflow

2017-03-17 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930246#comment-15930246 ] Thomas Groh commented on BEAM-1718: --- This is a Documentation/Implementation mismatch. I think that with

[jira] [Commented] (BEAM-1718) Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew() causes Overflow/Underflow

2017-03-20 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932906#comment-15932906 ] Thomas Groh commented on BEAM-1718: --- Most of the time, elements from a {{BoundedSource}} will be emitted

[jira] [Commented] (BEAM-1718) Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew() causes Overflow/Underflow

2017-03-20 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933005#comment-15933005 ] Thomas Groh commented on BEAM-1718: --- It is true for TextIO. "Most of the time" == Default behavior

[jira] [Created] (BEAM-1763) TestPipeline should ensure that all assertions succeeded

2017-03-20 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1763: - Summary: TestPipeline should ensure that all assertions succeeded Key: BEAM-1763 URL: https://issues.apache.org/jira/browse/BEAM-1763 Project: Beam Issue Type:

[jira] [Resolved] (BEAM-1732) Window.Assign does not properly populate DisplayData of the enclosing Window transform

2017-03-16 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1732. --- Resolution: Fixed Fix Version/s: First stable release > Window.Assign does not properly

[jira] [Created] (BEAM-1737) Interpreting a Single-output ParDo as a Multi-output ParDo with a single output causes serialization failures

2017-03-16 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1737: - Summary: Interpreting a Single-output ParDo as a Multi-output ParDo with a single output causes serialization failures Key: BEAM-1737 URL:

[jira] [Commented] (BEAM-1645) Display data not populated on Window.Assign

2017-03-16 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929093#comment-15929093 ] Thomas Groh commented on BEAM-1645: --- https://github.com/apache/beam/pull/2258 fixes this for

[jira] [Resolved] (BEAM-1776) Timers should be delivered in the window they were set in

2017-03-21 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1776. --- Resolution: Fixed Fix Version/s: First stable release > Timers should be delivered in the

[jira] [Created] (BEAM-1723) FlinkRunner should deduplicate when an UnboundedSource requires Deduping

2017-03-14 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1723: - Summary: FlinkRunner should deduplicate when an UnboundedSource requires Deduping Key: BEAM-1723 URL: https://issues.apache.org/jira/browse/BEAM-1723 Project: Beam

[jira] [Created] (BEAM-1724) ApexRunner should deduplicate when an UnboundedSource requires Deduping

2017-03-14 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1724: - Summary: ApexRunner should deduplicate when an UnboundedSource requires Deduping Key: BEAM-1724 URL: https://issues.apache.org/jira/browse/BEAM-1724 Project: Beam

[jira] [Created] (BEAM-1725) SparkRunner should deduplicate when an UnboundedSource requires Deduping

2017-03-14 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1725: - Summary: SparkRunner should deduplicate when an UnboundedSource requires Deduping Key: BEAM-1725 URL: https://issues.apache.org/jira/browse/BEAM-1725 Project: Beam

[jira] [Resolved] (BEAM-1869) getProducingTransformInternal should not be available on any PValue

2017-04-04 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1869. --- Resolution: Fixed Fix Version/s: First stable release > getProducingTransformInternal should

[jira] [Created] (BEAM-1928) Populate Runner API Components from the Java SDK

2017-04-10 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1928: - Summary: Populate Runner API Components from the Java SDK Key: BEAM-1928 URL: https://issues.apache.org/jira/browse/BEAM-1928 Project: Beam Issue Type: New

[jira] [Created] (BEAM-1930) Abstract Coder translation from Runner API Protos

2017-04-11 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1930: - Summary: Abstract Coder translation from Runner API Protos Key: BEAM-1930 URL: https://issues.apache.org/jira/browse/BEAM-1930 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-1931) The DirectRunner should garbage collect PCollectionView contents

2017-04-11 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1931: - Summary: The DirectRunner should garbage collect PCollectionView contents Key: BEAM-1931 URL: https://issues.apache.org/jira/browse/BEAM-1931 Project: Beam Issue

[jira] [Resolved] (BEAM-260) WindowMappingFn: Know the getSideInputWindow upper bound to release side input resources

2017-04-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-260. -- Resolution: Fixed Fix Version/s: First stable release > WindowMappingFn: Know the

[jira] [Created] (BEAM-1933) Revisit Runner API PCollection unique_name

2017-04-11 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1933: - Summary: Revisit Runner API PCollection unique_name Key: BEAM-1933 URL: https://issues.apache.org/jira/browse/BEAM-1933 Project: Beam Issue Type: Bug

[jira] [Assigned] (BEAM-819) The DirectRunner can complete before reporting the results of all ModelEnforcements

2017-04-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-819: Assignee: (was: Thomas Groh) > The DirectRunner can complete before reporting the results of all

[jira] [Assigned] (BEAM-436) DirectRunner RunnableOnService tempLocation configuration insufficient

2017-04-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-436: Assignee: (was: Thomas Groh) > DirectRunner RunnableOnService tempLocation configuration

[jira] [Resolved] (BEAM-1935) DirectRunner Cancel should never throw a RejectedExecutionException

2017-04-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1935. --- Resolution: Fixed Fix Version/s: First stable release > DirectRunner Cancel should never throw

[jira] [Resolved] (BEAM-1508) PInput, POutput#expand should not be ordered

2017-04-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1508. --- Resolution: Fixed Fix Version/s: First stable release > PInput, POutput#expand should not be

[jira] [Assigned] (BEAM-184) Using Merging Windows and/or Triggers without a downstream aggregation should fail

2017-04-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh reassigned BEAM-184: Assignee: (was: Thomas Groh) > Using Merging Windows and/or Triggers without a downstream

[jira] [Created] (BEAM-1935) DirectRunner Cancel should never throw a RejectedExecutionException

2017-04-11 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1935: - Summary: DirectRunner Cancel should never throw a RejectedExecutionException Key: BEAM-1935 URL: https://issues.apache.org/jira/browse/BEAM-1935 Project: Beam

[jira] [Commented] (BEAM-260) WindowMappingFn: Know the getSideInputWindow upper bound to release side input resources

2017-04-11 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964694#comment-15964694 ] Thomas Groh commented on BEAM-260: -- This is supported in the Java SDK. I've filed

[jira] [Created] (BEAM-1932) SideInputHandler should garbage collect PCollectionView contents

2017-04-11 Thread Thomas Groh (JIRA)
Thomas Groh created BEAM-1932: - Summary: SideInputHandler should garbage collect PCollectionView contents Key: BEAM-1932 URL: https://issues.apache.org/jira/browse/BEAM-1932 Project: Beam Issue

[jira] [Commented] (BEAM-1659) Improve UnboundedSourceShard structure

2017-04-03 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15953922#comment-15953922 ] Thomas Groh commented on BEAM-1659: --- Yeah, this seems as though it would be suitable. > Improve

[jira] [Updated] (BEAM-1138) Consider merging KeyedPValueTrackingVisitor with DirectGraphVisitor

2017-04-03 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh updated BEAM-1138: -- Labels: starter (was: ) > Consider merging KeyedPValueTrackingVisitor with DirectGraphVisitor >

  1   2   3   4   >