[jira] [Created] (BEAM-1197) Slowly-changing external data as a side input

2016-12-21 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-1197: -- Summary: Slowly-changing external data as a side input Key: BEAM-1197 URL: https://issues.apache.org/jira/browse/BEAM-1197 Project: Beam Issue Type: Wish

[jira] [Commented] (BEAM-1190) FileBasedSource should ignore files that matched the glob but don't exist

2016-12-21 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15767871#comment-15767871 ] Eugene Kirpichov commented on BEAM-1190: My proposal is to add a mandatory stat at

[jira] [Commented] (BEAM-1190) FileBasedSource should ignore files that matched the glob but don't exist

2016-12-20 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15766004#comment-15766004 ] Eugene Kirpichov commented on BEAM-1190: Dan - please note I'm suggesting not to ig

[jira] [Commented] (BEAM-1190) FileBasedSource should ignore files that matched the glob but don't exist

2016-12-20 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765658#comment-15765658 ] Eugene Kirpichov commented on BEAM-1190: CC: [~dhalp...@google.com] [~pei...@gmail.

[jira] [Created] (BEAM-1190) FileBasedSource should ignore files that matched the glob but don't exist

2016-12-20 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-1190: -- Summary: FileBasedSource should ignore files that matched the glob but don't exist Key: BEAM-1190 URL: https://issues.apache.org/jira/browse/BEAM-1190 Project: Be

[jira] [Created] (BEAM-1182) Direct runner should enforce encodability of unbounded source checkpoints

2016-12-19 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-1182: -- Summary: Direct runner should enforce encodability of unbounded source checkpoints Key: BEAM-1182 URL: https://issues.apache.org/jira/browse/BEAM-1182 Project: Be

[jira] [Resolved] (BEAM-1154) ReduceFnRunner fetches side input from the wrong window

2016-12-15 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov resolved BEAM-1154. Resolution: Fixed Fix Version/s: 0.4.0-incubating > ReduceFnRunner fetches side input

[jira] [Created] (BEAM-1159) DoFnSignature should have info on the fn's side inputs and outputs

2016-12-14 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-1159: -- Summary: DoFnSignature should have info on the fn's side inputs and outputs Key: BEAM-1159 URL: https://issues.apache.org/jira/browse/BEAM-1159 Project: Beam

[jira] [Created] (BEAM-1154) ReduceFnRunner fetches side input from the wrong window

2016-12-13 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-1154: -- Summary: ReduceFnRunner fetches side input from the wrong window Key: BEAM-1154 URL: https://issues.apache.org/jira/browse/BEAM-1154 Project: Beam Issue

[jira] [Resolved] (BEAM-1149) Side input access fails in direct runner (possibly others too) when input element in multiple windows

2016-12-13 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov resolved BEAM-1149. Resolution: Fixed Fix Version/s: 0.4.0-incubating > Side input access fails in direct

[jira] [Commented] (BEAM-1149) Side input access fails in direct runner (possibly others too) when input element in multiple windows

2016-12-13 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15746424#comment-15746424 ] Eugene Kirpichov commented on BEAM-1149: Test works with https://github.com/apache

[jira] [Created] (BEAM-1149) Side input access fails in direct runner (possibly others too) when input element in multiple windows

2016-12-13 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-1149: -- Summary: Side input access fails in direct runner (possibly others too) when input element in multiple windows Key: BEAM-1149 URL: https://issues.apache.org/jira/browse/BEAM-1

[jira] [Updated] (BEAM-863) WordCount on Windows: java.nio.file.InvalidPathException

2016-12-12 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov updated BEAM-863: -- Description: Running basic, unchanged WordCount on Windows with local files results in an excepti

[jira] [Created] (BEAM-1140) Apex ParDoBoundTranslatorTest.testMultiOutputParDoWithSideInputs is flaky

2016-12-12 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-1140: -- Summary: Apex ParDoBoundTranslatorTest.testMultiOutputParDoWithSideInputs is flaky Key: BEAM-1140 URL: https://issues.apache.org/jira/browse/BEAM-1140 Project: Be

[jira] [Commented] (BEAM-1134) Add support of String in MongoDBIO

2016-12-12 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742991#comment-15742991 ] Eugene Kirpichov commented on BEAM-1134: I agree that wasting like 10 lines writing

[jira] [Commented] (BEAM-1134) Add support of String in MongoDBIO

2016-12-12 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742741#comment-15742741 ] Eugene Kirpichov commented on BEAM-1134: I'm against making this part of MongoDbIO,

[jira] [Commented] (BEAM-1134) Add support of String in MongoDBIO

2016-12-12 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742598#comment-15742598 ] Eugene Kirpichov commented on BEAM-1134: Can this be done by simply composing the M

[jira] [Commented] (BEAM-801) SplittableParDo must be a pseudo-primitive, not ParDo(OldDoFn)

2016-12-01 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15713305#comment-15713305 ] Eugene Kirpichov commented on BEAM-801: --- This can be closed. > SplittableParDo must b

[jira] [Created] (BEAM-1060) Make DoFnTester use new DoFn

2016-11-29 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-1060: -- Summary: Make DoFnTester use new DoFn Key: BEAM-1060 URL: https://issues.apache.org/jira/browse/BEAM-1060 Project: Beam Issue Type: Improvement

[jira] [Created] (BEAM-988) Support for testing how soon output is emitted

2016-11-15 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-988: - Summary: Support for testing how soon output is emitted Key: BEAM-988 URL: https://issues.apache.org/jira/browse/BEAM-988 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-987) TestStream.advanceWatermarkToInfinity should perhaps also advance processing time

2016-11-15 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-987: - Summary: TestStream.advanceWatermarkToInfinity should perhaps also advance processing time Key: BEAM-987 URL: https://issues.apache.org/jira/browse/BEAM-987 Project

[jira] [Commented] (BEAM-896) FileBasedSink swap to temp directory broke Flink postcommit

2016-11-03 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634347#comment-15634347 ] Eugene Kirpichov commented on BEAM-896: --- Note that this is only a problem with local f

[jira] [Commented] (BEAM-896) FileBasedSink swap to temp directory broke Flink postcommit

2016-11-03 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634330#comment-15634330 ] Eugene Kirpichov commented on BEAM-896: --- The code is wrong because it doesn't delete t

[jira] [Closed] (BEAM-854) ReifyTimestampAndWindows explodes windows

2016-11-02 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov closed BEAM-854. - Resolution: Fixed Fix Version/s: 0.4.0-incubating ReifyTimestampsAndWindows no longer exists.

[jira] [Commented] (BEAM-801) SplittableParDo must be a pseudo-primitive, not ParDo(OldDoFn)

2016-11-02 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629943#comment-15629943 ] Eugene Kirpichov commented on BEAM-801: --- https://github.com/apache/incubator-beam/pull

[jira] [Commented] (BEAM-787) org.apache.beam.runners.direct.SplittableDoFnTest.testPairWithIndexBasic flaking

2016-11-02 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629942#comment-15629942 ] Eugene Kirpichov commented on BEAM-787: --- Will be fixed by https://github.com/apache/in

[jira] [Created] (BEAM-876) Support schemaUpdateOption in BigQueryIO

2016-11-01 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-876: - Summary: Support schemaUpdateOption in BigQueryIO Key: BEAM-876 URL: https://issues.apache.org/jira/browse/BEAM-876 Project: Beam Issue Type: Improvement

[jira] [Updated] (BEAM-854) ReifyTimestampAndWindows explodes windows

2016-10-27 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov updated BEAM-854: -- Priority: Major (was: Critical) > ReifyTimestampAndWindows explodes windows > ---

[jira] [Created] (BEAM-854) ReifyTimestampAndWindows explodes windows

2016-10-27 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-854: - Summary: ReifyTimestampAndWindows explodes windows Key: BEAM-854 URL: https://issues.apache.org/jira/browse/BEAM-854 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-787) org.apache.beam.runners.direct.SplittableDoFnTest.testPairWithIndexBasic flaking

2016-10-20 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593582#comment-15593582 ] Eugene Kirpichov commented on BEAM-787: --- I think I found the bug: https://github.com/

[jira] [Commented] (BEAM-787) org.apache.beam.runners.direct.SplittableDoFnTest.testPairWithIndexBasic flaking

2016-10-20 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593044#comment-15593044 ] Eugene Kirpichov commented on BEAM-787: --- I'm able to reproduce this locally in a few s

[jira] [Commented] (BEAM-638) Add a Window function to create a bounded PCollection from an unbounded one

2016-10-14 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15575679#comment-15575679 ] Eugene Kirpichov commented on BEAM-638: --- More discussion currently happening on http:

[jira] [Created] (BEAM-750) Remove StateTag

2016-10-13 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-750: - Summary: Remove StateTag Key: BEAM-750 URL: https://issues.apache.org/jira/browse/BEAM-750 Project: Beam Issue Type: Improvement Reporter: Euge

[jira] [Commented] (BEAM-714) Migrate DatastoreV1 to use AutoValue to reduce boilerplate

2016-10-12 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15570357#comment-15570357 ] Eugene Kirpichov commented on BEAM-714: --- Should be closed. > Migrate DatastoreV1 to u

[jira] [Assigned] (BEAM-714) Migrate DatastoreV1 to use AutoValue to reduce boilerplate

2016-10-12 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov reassigned BEAM-714: - Assignee: Eugene Kirpichov (was: Daniel Halperin) > Migrate DatastoreV1 to use AutoValue t

[jira] [Created] (BEAM-730) Remove Reshuffle transform in favor of Redistribute.byKey()

2016-10-07 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-730: - Summary: Remove Reshuffle transform in favor of Redistribute.byKey() Key: BEAM-730 URL: https://issues.apache.org/jira/browse/BEAM-730 Project: Beam Issue

[jira] [Created] (BEAM-729) Does sdks/java/io/google-cloud-platform/pom.xml need javadoc plugin?

2016-10-07 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-729: - Summary: Does sdks/java/io/google-cloud-platform/pom.xml need javadoc plugin? Key: BEAM-729 URL: https://issues.apache.org/jira/browse/BEAM-729 Project: Beam

[jira] [Commented] (BEAM-717) Migrate KafkaIO to use AutoValue to reduce boilerplate

2016-10-06 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15552714#comment-15552714 ] Eugene Kirpichov commented on BEAM-717: --- https://github.com/apache/incubator-beam/pull

[jira] [Assigned] (BEAM-717) Migrate KafkaIO to use AutoValue to reduce boilerplate

2016-10-06 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov reassigned BEAM-717: - Assignee: Eugene Kirpichov (was: James Malone) > Migrate KafkaIO to use AutoValue to reduc

[jira] [Commented] (BEAM-714) Migrate DatastoreV1 to use AutoValue to reduce boilerplate

2016-10-05 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549629#comment-15549629 ] Eugene Kirpichov commented on BEAM-714: --- This is being done in https://github.com/apac

[jira] [Commented] (BEAM-702) Simple pattern for per-bundle and per-DoFn Closeable resources

2016-10-04 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546300#comment-15546300 ] Eugene Kirpichov commented on BEAM-702: --- Hmm, I didn't realize that we don't call fini

[jira] [Commented] (BEAM-702) Simple pattern for per-bundle and per-DoFn Closeable resources

2016-10-03 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543109#comment-15543109 ] Eugene Kirpichov commented on BEAM-702: --- CC: [~tgroh] > Simple pattern for per-bundle

[jira] [Created] (BEAM-702) Simple pattern for per-bundle and per-DoFn Closeable resources

2016-10-03 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-702: - Summary: Simple pattern for per-bundle and per-DoFn Closeable resources Key: BEAM-702 URL: https://issues.apache.org/jira/browse/BEAM-702 Project: Beam Is

[jira] [Created] (BEAM-694) TriggerTester doesn't test timer firings

2016-09-29 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-694: - Summary: TriggerTester doesn't test timer firings Key: BEAM-694 URL: https://issues.apache.org/jira/browse/BEAM-694 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-559) DoFnTester should handle Setup / TearDown

2016-09-27 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527667#comment-15527667 ] Eugene Kirpichov commented on BEAM-559: --- This issue should be closed now. > DoFnTeste

[jira] [Commented] (BEAM-559) DoFnTester should handle Setup / TearDown

2016-09-26 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524677#comment-15524677 ] Eugene Kirpichov commented on BEAM-559: --- PR https://github.com/apache/incubator-beam/p

[jira] [Commented] (BEAM-630) DataflowRunnableOnService test suite is broken

2016-09-13 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488969#comment-15488969 ] Eugene Kirpichov commented on BEAM-630: --- Hmm, looks like I don't have the rights to cl

[jira] [Commented] (BEAM-630) DataflowRunnableOnService test suite is broken

2016-09-13 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488439#comment-15488439 ] Eugene Kirpichov commented on BEAM-630: --- WELL, ACTUALLY it might be our bug. Almost na

[jira] [Comment Edited] (BEAM-630) DataflowRunnableOnService test suite is broken

2016-09-13 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488397#comment-15488397 ] Eugene Kirpichov edited comment on BEAM-630 at 9/13/16 9:01 PM: --

[jira] [Comment Edited] (BEAM-630) DataflowRunnableOnService test suite is broken

2016-09-13 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488397#comment-15488397 ] Eugene Kirpichov edited comment on BEAM-630 at 9/13/16 8:57 PM: --

[jira] [Commented] (BEAM-630) DataflowRunnableOnService test suite is broken

2016-09-13 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488397#comment-15488397 ] Eugene Kirpichov commented on BEAM-630: --- Wonderful. This seems like a bug in ByteBuddy

[jira] [Commented] (BEAM-630) DataflowRunnableOnService test suite is broken

2016-09-13 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488356#comment-15488356 ] Eugene Kirpichov commented on BEAM-630: --- Some investigation: - The issue can be narrow

[jira] [Commented] (BEAM-65) SplittableDoFn

2016-08-04 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-65?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408251#comment-15408251 ] Eugene Kirpichov commented on BEAM-65: -- Proposal announced on Beam dev mailing list htt

[jira] [Created] (BEAM-521) Execute some file-based reads via composite transform instead of custom source

2016-08-03 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-521: - Summary: Execute some file-based reads via composite transform instead of custom source Key: BEAM-521 URL: https://issues.apache.org/jira/browse/BEAM-521 Project: B

[jira] [Updated] (BEAM-521) Execute some bounded source reads via composite transform

2016-08-03 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov updated BEAM-521: -- Summary: Execute some bounded source reads via composite transform (was: Execute some file-based

[jira] [Commented] (BEAM-217) BoundedSource.splitAtFraction should be splitAfterFraction

2016-05-06 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15274765#comment-15274765 ] Eugene Kirpichov commented on BEAM-217: --- Seems like this is becoming more important as

[jira] [Commented] (BEAM-217) BoundedSource.splitAtFraction should be splitAfterFraction

2016-04-25 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257180#comment-15257180 ] Eugene Kirpichov commented on BEAM-217: --- I'm overwhelmed and don't have time to do it

[jira] [Commented] (BEAM-215) Create should be implemented as a BoundedSource

2016-04-21 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15252038#comment-15252038 ] Eugene Kirpichov commented on BEAM-215: --- Note that for this to interact well with Data

[jira] [Updated] (BEAM-217) BoundedSource.splitAtFraction should be splitAfterFraction

2016-04-21 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov updated BEAM-217: -- Priority: Minor (was: Major) > BoundedSource.splitAtFraction should be splitAfterFraction > -

[jira] [Created] (BEAM-217) BoundedSource.splitAtFraction should be splitAfterFraction

2016-04-21 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-217: - Summary: BoundedSource.splitAtFraction should be splitAfterFraction Key: BEAM-217 URL: https://issues.apache.org/jira/browse/BEAM-217 Project: Beam Issue T

[jira] [Resolved] (BEAM-182) SourceTestUtils.assertSplitAtFractionExhaustive should do a bounded number of trials

2016-04-07 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov resolved BEAM-182. --- Resolution: Fixed > SourceTestUtils.assertSplitAtFractionExhaustive should do a bounded number o

[jira] [Commented] (BEAM-182) SourceTestUtils.assertSplitAtFractionExhaustive should do a bounded number of trials

2016-04-07 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15230963#comment-15230963 ] Eugene Kirpichov commented on BEAM-182: --- https://github.com/apache/incubator-beam/pull

[jira] [Created] (BEAM-182) SourceTestUtils.assertSplitAtFractionExhaustive should do a bounded number of trials

2016-04-07 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-182: - Summary: SourceTestUtils.assertSplitAtFractionExhaustive should do a bounded number of trials Key: BEAM-182 URL: https://issues.apache.org/jira/browse/BEAM-182 Proj

[jira] [Updated] (BEAM-167) TextIO can't read concatenated gzip files

2016-04-04 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov updated BEAM-167: -- Component/s: sdk-java-core > TextIO can't read concatenated gzip files > -

[jira] [Commented] (BEAM-167) TextIO can't read concatenated gzip files

2016-04-04 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224834#comment-15224834 ] Eugene Kirpichov commented on BEAM-167: --- Here's a test and a patch https://gist.githu

[jira] [Created] (BEAM-167) TextIO can't read concatenated gzip files

2016-04-04 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-167: - Summary: TextIO can't read concatenated gzip files Key: BEAM-167 URL: https://issues.apache.org/jira/browse/BEAM-167 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-47) UnboundedSource should support dynamic work rebalancing

2016-04-01 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221972#comment-15221972 ] Eugene Kirpichov commented on BEAM-47: -- Is this a dupe of BEAM-65? > UnboundedSource sh

[jira] [Commented] (BEAM-68) Support for limiting parallelism of a step

2016-03-31 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220089#comment-15220089 ] Eugene Kirpichov commented on BEAM-68: -- I re-duped BEAM-169 against BEAM-92. - I agree t

[jira] [Closed] (BEAM-159) Support fixed number of shards in custom sinks

2016-03-31 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov closed BEAM-159. - Resolution: Duplicate > Support fixed number of shards in custom sinks > ---

[jira] [Reopened] (BEAM-159) Support fixed number of shards in custom sinks

2016-03-31 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Kirpichov reopened BEAM-159: --- Reopening to dupe against BEAM-92 instead, per discussion in BEAM-68. > Support fixed number of sha

[jira] [Commented] (BEAM-68) Support for limiting parallelism of a step

2016-03-30 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219126#comment-15219126 ] Eugene Kirpichov commented on BEAM-68: -- I think the two use cases are quite different an

[jira] [Created] (BEAM-159) Support fixed number of shards in custom sinks

2016-03-30 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-159: - Summary: Support fixed number of shards in custom sinks Key: BEAM-159 URL: https://issues.apache.org/jira/browse/BEAM-159 Project: Beam Issue Type: Improve

[jira] [Created] (BEAM-153) Support timeout in runner API

2016-03-29 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-153: - Summary: Support timeout in runner API Key: BEAM-153 URL: https://issues.apache.org/jira/browse/BEAM-153 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-92) Data-dependent sinks

2016-03-03 Thread Eugene Kirpichov (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178147#comment-15178147 ] Eugene Kirpichov commented on BEAM-92: -- Note: it's not obvious how this API generalizes

[jira] [Created] (BEAM-92) Data-dependent sinks

2016-03-03 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-92: Summary: Data-dependent sinks Key: BEAM-92 URL: https://issues.apache.org/jira/browse/BEAM-92 Project: Beam Issue Type: New Feature Reporter: Eug

[jira] [Created] (BEAM-89) DataflowPipelineJob should have an API that prints messages but doesn't wait for completion

2016-03-02 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-89: Summary: DataflowPipelineJob should have an API that prints messages but doesn't wait for completion Key: BEAM-89 URL: https://issues.apache.org/jira/browse/BEAM-89 P

[jira] [Created] (BEAM-88) DataflowPipelineOptions.tempLocation doesn't really default to stagingLocation

2016-03-02 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-88: Summary: DataflowPipelineOptions.tempLocation doesn't really default to stagingLocation Key: BEAM-88 URL: https://issues.apache.org/jira/browse/BEAM-88 Project: Beam