[jira] [Assigned] (BEAM-5760) Portable Flink support for maxBundleSize/maxBundleMillis

2018-10-16 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reassigned BEAM-5760: -- Assignee: Thomas Weise > Portable Flink support for maxBundleSize/maxBundleMillis >

[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job issues

2018-10-21 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16658492#comment-16658492 ] Thomas Weise commented on BEAM-5467: [~swegner] thanks for taking a look. There have been a couple

[jira] [Updated] (BEAM-5467) Python Flink ValidatesRunner job issues

2018-10-21 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5467: --- Summary: Python Flink ValidatesRunner job issues (was: Python Flink ValidatesRunner job fixes) >

[jira] [Updated] (BEAM-5467) Python Flink ValidatesRunner job issues

2018-10-21 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5467: --- Description:   The job is currently flaky and does not provide a useful signal.  was: Add status

[jira] [Resolved] (BEAM-5797) SDK workers are not always killed when Flink pipeline finishes

2018-10-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-5797. Resolution: Fixed Assignee: Micah Wylde Fix Version/s: 2.9.0 > SDK workers are not

[jira] [Commented] (BEAM-5816) Flink portable bundles can fail to close with "Already closed" exception

2018-10-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16659697#comment-16659697 ] Thomas Weise commented on BEAM-5816: Underlying (larger) issue is that DoFnOperator.dispose can

[jira] [Updated] (BEAM-5793) Python sdk docs target fails in flink_streaming_impulse_source

2018-10-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5793: --- Fix Version/s: (was: 2.8.0) Not applicable > Python sdk docs target fails in

[jira] [Commented] (BEAM-5397) Flink portable runner GRPC cleanup failure after user class loader was removed

2018-10-19 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657290#comment-16657290 ] Thomas Weise commented on BEAM-5397: Yes, this is still relevant with current master / 2.9.0-SNAPSHOT.

[jira] [Resolved] (BEAM-5760) Portable Flink support for maxBundleSize/maxBundleMillis

2018-10-20 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-5760. Resolution: Fixed > Portable Flink support for maxBundleSize/maxBundleMillis >

[jira] [Updated] (BEAM-5816) Flink runner starts new bundles while disposing operator

2018-10-23 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5816: --- Summary: Flink runner starts new bundles while disposing operator (was: Flink portable bundles can

[jira] [Commented] (BEAM-5826) Vendor slf4j API

2018-10-23 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660808#comment-16660808 ] Thomas Weise commented on BEAM-5826: I don't think that we can vendor slf4j. This would prevent users

[jira] [Resolved] (BEAM-5848) Flink runner synthetic source with forward stream causes job failure

2018-10-24 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-5848. Resolution: Fixed Fix Version/s: 2.9.0 > Flink runner synthetic source with forward stream

[jira] [Commented] (BEAM-5026) Portable flink wordcount fails sometimes due to non-existent source path in FileBasedSink._check_state_for_finalize_write

2018-11-01 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671616#comment-16671616 ] Thomas Weise commented on BEAM-5026: The reason for this is the use of multiple docker containers due

[jira] [Resolved] (BEAM-5708) Support caching of SDKHarness environments in flink

2018-11-01 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-5708. Resolution: Fixed Fix Version/s: 2.9.0 > Support caching of SDKHarness environments in flink

[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job issues

2018-11-02 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673814#comment-16673814 ] Thomas Weise commented on BEAM-5467: Looks like the same issue to me, if the seg fault occurs after

[jira] [Closed] (BEAM-5916) Python PostCommit python:flinkCompatibilityMatrixBatchDOCKER fails

2018-11-02 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise closed BEAM-5916. -- Resolution: Duplicate Assignee: (was: Ankur Goenka) Fix Version/s: Not applicable >

[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job issues

2018-11-05 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675253#comment-16675253 ] Thomas Weise commented on BEAM-5467: I did a little more investigation here:

[jira] [Created] (BEAM-5848) Sending empty byte arrays through forward stream causes Flink job failure

2018-10-23 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-5848: -- Summary: Sending empty byte arrays through forward stream causes Flink job failure Key: BEAM-5848 URL: https://issues.apache.org/jira/browse/BEAM-5848 Project: Beam

[jira] [Updated] (BEAM-5848) Sending empty byte arrays through forward stream causes Flink job failure

2018-10-23 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5848: --- Attachment: image-2018-10-23-20-46-58-991.png > Sending empty byte arrays through forward stream

[jira] [Updated] (BEAM-5848) Flink runner synthetic source with forward stream causes job failure

2018-10-23 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5848: --- Summary: Flink runner synthetic source with forward stream causes job failure (was: Flink synthetic

[jira] [Updated] (BEAM-5848) Flink synthetic source with forward stream causes job failure

2018-10-23 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5848: --- Summary: Flink synthetic source with forward stream causes job failure (was: Sending empty byte

[jira] [Updated] (BEAM-2863) Add support for Side Inputs over the Fn API

2018-10-29 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-2863: --- Fix Version/s: (was: 2.8.0) Not applicable > Add support for Side Inputs over

[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job issues

2018-11-09 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682139#comment-16682139 ] Thomas Weise commented on BEAM-5467: The culprit is actually

[jira] [Created] (BEAM-6034) Add option to set the Flink auto watermarking interval

2018-11-10 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6034: -- Summary: Add option to set the Flink auto watermarking interval Key: BEAM-6034 URL: https://issues.apache.org/jira/browse/BEAM-6034 Project: Beam Issue Type:

[jira] [Commented] (BEAM-6058) Support flink config directory for flink runner.

2018-11-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688253#comment-16688253 ] Thomas Weise commented on BEAM-6058: I don't see why old Flink runner and job server server should

[jira] [Resolved] (BEAM-6034) Add option to set the Flink auto watermarking interval

2018-11-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-6034. Resolution: Fixed Fix Version/s: 2.9.0 > Add option to set the Flink auto watermarking

[jira] [Created] (BEAM-6440) FlinkTimerInternals memory leak

2019-01-15 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6440: -- Summary: FlinkTimerInternals memory leak Key: BEAM-6440 URL: https://issues.apache.org/jira/browse/BEAM-6440 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-6440) FlinkTimerInternals memory leak

2019-01-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743291#comment-16743291 ] Thomas Weise commented on BEAM-6440: !image-2019-01-15-10-41-08-200.png|width=100%! >

[jira] [Updated] (BEAM-6440) FlinkTimerInternals memory leak

2019-01-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-6440: --- Attachment: image-2019-01-15-10-31-43-618.png > FlinkTimerInternals memory leak >

[jira] [Updated] (BEAM-6440) FlinkTimerInternals memory leak

2019-01-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-6440: --- Attachment: image-2019-01-15-10-41-08-200.png > FlinkTimerInternals memory leak >

[jira] [Commented] (BEAM-6440) FlinkTimerInternals memory leak

2019-01-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743285#comment-16743285 ] Thomas Weise commented on BEAM-6440: !image-2019-01-15-10-31-43-618.png|width=100%! >

[jira] [Commented] (BEAM-6294) Use Flink's redistribute for reshuffle.

2019-01-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743484#comment-16743484 ] Thomas Weise commented on BEAM-6294: [~robertwb] [~mxm] we found that the non-native reshuffle

[jira] [Updated] (BEAM-6440) FlinkTimerInternals memory leak

2019-01-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-6440: --- Issue Type: Bug (was: Improvement) > FlinkTimerInternals memory leak >

[jira] [Commented] (BEAM-3272) ParDoTranslatorTest: Error creating local cluster while creating checkpoint file

2019-01-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743520#comment-16743520 ] Thomas Weise commented on BEAM-3272: FWIW the unique directory issue was fixed. >

[jira] [Updated] (BEAM-6440) FlinkTimerInternals memory leak

2019-01-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-6440: --- Fix Version/s: 2.10.0 > FlinkTimerInternals memory leak > --- > >

[jira] [Commented] (BEAM-6440) FlinkTimerInternals memory leak

2019-01-15 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743562#comment-16743562 ] Thomas Weise commented on BEAM-6440: Verified changes with our application and heap utilization is

[jira] [Assigned] (BEAM-5442) PortableRunner swallows custom options for Runner

2019-01-17 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reassigned BEAM-5442: -- Assignee: Thomas Weise (was: Maximilian Michels) > PortableRunner swallows custom options for

[jira] [Commented] (BEAM-5396) Flink portable runner savepoint / upgrade support

2018-12-12 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719435#comment-16719435 ] Thomas Weise commented on BEAM-5396: The pipeline option to specify savepoint restore is now in place

[jira] [Resolved] (BEAM-6165) Send metrics to Flink in portable Flink runner

2018-12-20 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-6165. Resolution: Fixed Fix Version/s: 2.10.0 > Send metrics to Flink in portable Flink runner >

[jira] [Commented] (BEAM-6242) PreCommit Portable WordCount is flaky

2018-12-17 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723059#comment-16723059 ] Thomas Weise commented on BEAM-6242: It may be related to [https://github.com/apache/beam/pull/7174]  

[jira] [Commented] (BEAM-6242) PreCommit Portable WordCount is flaky

2018-12-17 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723058#comment-16723058 ] Thomas Weise commented on BEAM-6242: Failure rate is up since Dec 12. 

[jira] [Resolved] (BEAM-6227) FlinkRunner errors if GroupByKey contains null values (streaming mode only)

2018-12-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-6227. Resolution: Fixed > FlinkRunner errors if GroupByKey contains null values (streaming mode only) >

[jira] [Created] (BEAM-6230) Document Flink version support policy

2018-12-13 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6230: -- Summary: Document Flink version support policy Key: BEAM-6230 URL: https://issues.apache.org/jira/browse/BEAM-6230 Project: Beam Issue Type: Improvement

[jira] [Assigned] (BEAM-6230) Document Flink version support policy

2018-12-13 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reassigned BEAM-6230: -- Assignee: (was: Melissa Pashniak) > Document Flink version support policy >

[jira] [Updated] (BEAM-5396) Flink portable runner savepoint / upgrade support

2018-11-28 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5396: --- Description: The portable Flink runner needs to support Flink savepoints for production use. It

[jira] [Commented] (BEAM-5396) Flink portable runner savepoint / upgrade support

2018-11-28 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702791#comment-16702791 ] Thomas Weise commented on BEAM-5396: I started work to add the pipeline options. But as of Flink

[jira] [Commented] (BEAM-5167) Use concurrency information from SDK Harness in Flink Portable Runner

2018-11-30 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16705596#comment-16705596 ] Thomas Weise commented on BEAM-5167: Working combination for streaming: {code:java} --streaming

[jira] [Comment Edited] (BEAM-5167) Use concurrency information from SDK Harness in Flink Portable Runner

2018-11-30 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16705594#comment-16705594 ] Thomas Weise edited comment on BEAM-5167 at 12/1/18 3:33 AM: - The issue can be

[jira] [Closed] (BEAM-6122) Update committer guidelines

2018-12-04 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise closed BEAM-6122. -- Resolution: Done Fix Version/s: Not applicable > Update committer guidelines >

[jira] [Resolved] (BEAM-6163) Support Process environment on Mac

2018-12-01 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-6163. Resolution: Fixed Fix Version/s: 2.10.0 > Support Process environment on Mac >

[jira] [Created] (BEAM-6147) Python process environment factory

2018-11-29 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6147: -- Summary: Python process environment factory Key: BEAM-6147 URL: https://issues.apache.org/jira/browse/BEAM-6147 Project: Beam Issue Type: Task

[jira] [Commented] (BEAM-6147) Python process environment factory

2018-11-29 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703350#comment-16703350 ] Thomas Weise commented on BEAM-6147: Attempt to enable process worker for post-commit:

[jira] [Commented] (BEAM-6147) Python process environment factory

2018-11-29 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703354#comment-16703354 ] Thomas Weise commented on BEAM-6147: At Lyft we implemented a custom factory that is tailored to the

[jira] [Assigned] (BEAM-6111) org.apache.beam.runners.flink.PortableTimersExecutionTest is very flakey 40/50 runs failed

2018-11-21 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reassigned BEAM-6111: -- Assignee: Maximilian Michels > org.apache.beam.runners.flink.PortableTimersExecutionTest is

[jira] [Commented] (BEAM-5419) Build multiple versions of the Flink Runner against different Flink versions

2018-11-24 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697919#comment-16697919 ] Thomas Weise commented on BEAM-5419: We should provide binaries for 1.6.x for the next release. I see

[jira] [Commented] (BEAM-5419) Build multiple versions of the Flink Runner against different Flink versions

2018-11-24 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697920#comment-16697920 ] Thomas Weise commented on BEAM-5419: Mailing list thread:

[jira] [Updated] (BEAM-5440) Add option to mount a directory inside SDK harness containers

2018-11-21 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5440: --- Fix Version/s: (was: 3.0.0) > Add option to mount a directory inside SDK harness containers >

[jira] [Commented] (BEAM-5440) Add option to mount a directory inside SDK harness containers

2018-11-21 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694743#comment-16694743 ] Thomas Weise commented on BEAM-5440: Removing fix version since convention in BEAM is to set it when

[jira] [Created] (BEAM-6122) Update committer guidelines

2018-11-25 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6122: -- Summary: Update committer guidelines Key: BEAM-6122 URL: https://issues.apache.org/jira/browse/BEAM-6122 Project: Beam Issue Type: Task Components:

[jira] [Closed] (BEAM-1400) Transient Apex runner postcommit failure in WindowTest.testOutputTimeFnDefault

2018-11-27 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise closed BEAM-1400. -- Resolution: Cannot Reproduce Fix Version/s: Not applicable I don't think this is an issue any

[jira] [Updated] (BEAM-5467) Python Flink ValidatesRunner job issues

2018-11-27 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5467: --- Labels: flake portability-flink (was: portability-flink) > Python Flink ValidatesRunner job issues >

[jira] [Resolved] (BEAM-5197) Flaky test: UnboundedSourceWrapperTest$ParameterizedUnboundedSourceWrapperTest

2018-11-27 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-5197. Resolution: Fixed > Flaky test: UnboundedSourceWrapperTest$ParameterizedUnboundedSourceWrapperTest

[jira] [Resolved] (BEAM-6058) Support flink config directory for flink runner.

2018-11-27 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-6058. Resolution: Fixed Fix Version/s: 2.10.0 > Support flink config directory for flink runner. >

[jira] [Commented] (BEAM-6116) Pushed back watermark never emitted in portable Flink runner

2018-11-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696167#comment-16696167 ] Thomas Weise commented on BEAM-6116: Fix and changes to reproduce:

[jira] [Commented] (BEAM-6116) Pushed back watermark never emitted in portable Flink runner

2018-11-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696172#comment-16696172 ] Thomas Weise commented on BEAM-6116: [~chamikara] The fix should also go to the 2.9 release branch

[jira] [Created] (BEAM-6116) Pushed back watermark never emitted in portable Flink runner

2018-11-22 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6116: -- Summary: Pushed back watermark never emitted in portable Flink runner Key: BEAM-6116 URL: https://issues.apache.org/jira/browse/BEAM-6116 Project: Beam Issue

[jira] [Comment Edited] (BEAM-6116) Pushed back watermark never emitted in portable Flink runner

2018-11-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696172#comment-16696172 ] Thomas Weise edited comment on BEAM-6116 at 11/22/18 6:19 PM: -- [~chamikara]

[jira] [Updated] (BEAM-5442) PortableRunner swallows custom options for Runner

2018-11-21 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-5442: --- Fix Version/s: (was: 3.0.0) 2.10.0 > PortableRunner swallows custom options

[jira] [Assigned] (BEAM-5396) Flink portable runner savepoint / upgrade support

2018-11-28 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reassigned BEAM-5396: -- Assignee: Thomas Weise > Flink portable runner savepoint / upgrade support >

[jira] [Commented] (BEAM-6294) Use Flink's redistribute for reshuffle.

2019-01-11 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740879#comment-16740879 ] Thomas Weise commented on BEAM-6294: Yes, this is fixed: [https://github.com/apache/beam/pull/7490]  

[jira] [Resolved] (BEAM-6294) Use Flink's redistribute for reshuffle.

2019-01-11 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-6294. Resolution: Fixed > Use Flink's redistribute for reshuffle. >

[jira] [Reopened] (BEAM-6294) Use Flink's redistribute for reshuffle.

2019-01-09 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reopened BEAM-6294: Reopening the issue since it breaks our application: * The PR contains substantial changes but no test

[jira] [Commented] (BEAM-6294) Use Flink's redistribute for reshuffle.

2019-01-09 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738470#comment-16738470 ] Thomas Weise commented on BEAM-6294: Error that shows up shortly after job enters running status:

[jira] [Commented] (BEAM-6294) Use Flink's redistribute for reshuffle.

2019-01-09 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738478#comment-16738478 ] Thomas Weise commented on BEAM-6294: The issue is specifically caused by the reshuffle transform after

[jira] [Commented] (BEAM-6294) Use Flink's redistribute for reshuffle.

2019-01-09 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738701#comment-16738701 ] Thomas Weise commented on BEAM-6294: Inserting shuffle into the synthetic source example: {code:java}

[jira] [Commented] (BEAM-6924) Prevent overflow when Flink bundle timeout is Long.MAX_VALUE

2019-03-27 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803364#comment-16803364 ] Thomas Weise commented on BEAM-6924: How did you find this?   > Prevent overflow when Flink bundle

[jira] [Commented] (BEAM-6993) Default to cleaning staged artifacts after running portable pipeline

2019-04-03 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808979#comment-16808979 ] Thomas Weise commented on BEAM-6993: You probably mean after the pipeline finished vs. after

[jira] [Commented] (BEAM-6753) Create proto representation for schemas

2019-04-05 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811068#comment-16811068 ] Thomas Weise commented on BEAM-6753: [~reuvenlax] seeing that the PR was merged, could you add a bit

[jira] [Created] (BEAM-6829) Duplicate metric warnings clutter log

2019-03-13 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6829: -- Summary: Duplicate metric warnings clutter log Key: BEAM-6829 URL: https://issues.apache.org/jira/browse/BEAM-6829 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-6829) Duplicate metric warnings clutter log

2019-03-13 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792335#comment-16792335 ] Thomas Weise commented on BEAM-6829: {code:java} [[1]Create/FlatMap() (5/8)] WARN

[jira] [Work started] (BEAM-6796) Portable Flink runner error in finish bundle timer processing

2019-03-09 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on BEAM-6796 started by Thomas Weise. -- > Portable Flink runner error in finish bundle timer processing >

[jira] [Created] (BEAM-6796) Portable Flink runner error in finish bundle timer processing

2019-03-09 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6796: -- Summary: Portable Flink runner error in finish bundle timer processing Key: BEAM-6796 URL: https://issues.apache.org/jira/browse/BEAM-6796 Project: Beam Issue

[jira] [Created] (BEAM-6879) Portable runner support for @RequiresStableInput

2019-03-21 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6879: -- Summary: Portable runner support for @RequiresStableInput Key: BEAM-6879 URL: https://issues.apache.org/jira/browse/BEAM-6879 Project: Beam Issue Type: Task

[jira] [Created] (BEAM-6876) User state cleanup in portable Flink runner

2019-03-21 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6876: -- Summary: User state cleanup in portable Flink runner Key: BEAM-6876 URL: https://issues.apache.org/jira/browse/BEAM-6876 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-6640) Reshuffle not translated to Flink rebalance

2019-02-09 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6640: -- Summary: Reshuffle not translated to Flink rebalance Key: BEAM-6640 URL: https://issues.apache.org/jira/browse/BEAM-6640 Project: Beam Issue Type: Task

[jira] [Resolved] (BEAM-5816) Flink runner starts new bundles while disposing operator

2019-02-13 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-5816. Resolution: Fixed Fix Version/s: 2.11.0 > Flink runner starts new bundles while disposing

[jira] [Updated] (BEAM-3272) ParDoTranslatorTest: Error creating local cluster while creating checkpoint file

2019-02-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-3272: --- Fix Version/s: (was: 2.11.0) > ParDoTranslatorTest: Error creating local cluster while creating

[jira] [Updated] (BEAM-3272) ParDoTranslatorTest: Error creating local cluster while creating checkpoint file

2019-02-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-3272: --- Priority: Minor (was: Critical) > ParDoTranslatorTest: Error creating local cluster while creating

[jira] [Created] (BEAM-6676) Python timers only working with GlobalWindow

2019-02-14 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6676: -- Summary: Python timers only working with GlobalWindow Key: BEAM-6676 URL: https://issues.apache.org/jira/browse/BEAM-6676 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-6676) Python timers only working with GlobalWindow

2019-02-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768916#comment-16768916 ] Thomas Weise commented on BEAM-6676: {code:java} from __future__ import absolute_import import

[jira] [Commented] (BEAM-6676) Python timers only working with GlobalWindow

2019-02-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768918#comment-16768918 ] Thomas Weise commented on BEAM-6676: Here is the error: {code:java}     timer.set(1000)   File

[jira] [Resolved] (BEAM-2687) Python SDK support for Stateful Processing

2019-02-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-2687. Resolution: Implemented Fix Version/s: 2.9.0 > Python SDK support for Stateful Processing >

[jira] [Created] (BEAM-6537) Separate runner and SDK options

2019-01-29 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6537: -- Summary: Separate runner and SDK options Key: BEAM-6537 URL: https://issues.apache.org/jira/browse/BEAM-6537 Project: Beam Issue Type: Task

[jira] [Resolved] (BEAM-5442) PortableRunner swallows custom options for Runner

2019-01-29 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise resolved BEAM-5442. Resolution: Fixed Fix Version/s: 2.11.0 > PortableRunner swallows custom options for Runner

[jira] [Commented] (BEAM-6537) Separate runner and SDK options

2019-01-29 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755277#comment-16755277 ] Thomas Weise commented on BEAM-6537: For more context see:

[jira] [Created] (BEAM-6531) Namespace for pipeline options

2019-01-28 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-6531: -- Summary: Namespace for pipeline options Key: BEAM-6531 URL: https://issues.apache.org/jira/browse/BEAM-6531 Project: Beam Issue Type: Task Components:

[jira] [Closed] (BEAM-2889) Flink runs portable pipelines

2019-02-02 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise closed BEAM-2889. -- Resolution: Fixed Assignee: (was: Ben Sidhom) Fix Version/s: 2.10.0 > Flink runs

[jira] [Commented] (BEAM-7035) Clear() method of OutputTimer is inconsistent

2019-04-08 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812906#comment-16812906 ] Thomas Weise commented on BEAM-7035: This change is necessary but I don't think clearing a timer can

[jira] [Created] (BEAM-7058) Python SDK metric process_bundle_msecs reported as zero

2019-04-11 Thread Thomas Weise (JIRA)
Thomas Weise created BEAM-7058: -- Summary: Python SDK metric process_bundle_msecs reported as zero Key: BEAM-7058 URL: https://issues.apache.org/jira/browse/BEAM-7058 Project: Beam Issue Type:

[jira] [Assigned] (BEAM-7015) Have only a single definition of standard_coders.yaml

2019-04-16 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reassigned BEAM-7015: -- Assignee: Thomas Weise (was: Robert Bradshaw) > Have only a single definition of

  1   2   3   >