[jira] [Commented] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-12 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16647630#comment-16647630 ] Maximilian Michels commented on BEAM-5713: -- {quote} I added more tasks and they are all squeezed

[jira] [Commented] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646815#comment-16646815 ] Maximilian Michels commented on BEAM-5713: -- After testing this with native Flink jobs, looking

[jira] [Updated] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5713: - Attachment: Different SlotSharingGroup.png > Flink portable runner schedules all tasks of

[jira] [Commented] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646497#comment-16646497 ] Maximilian Michels commented on BEAM-5713: -- Putting the native source into a different

[jira] [Commented] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646479#comment-16646479 ] Maximilian Michels commented on BEAM-5713: -- Seems like this was largely by chance because your

[jira] [Commented] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646345#comment-16646345 ] Maximilian Michels commented on BEAM-5713: -- !With RichParallelSourceFunction and parallelism

[jira] [Updated] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5713: - Attachment: With RichParallelSourceFunction and parallelism 5.png > Flink portable runner

[jira] [Commented] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646343#comment-16646343 ] Maximilian Michels commented on BEAM-5713: -- Actually, this is a FlinkRunner bug. The Impulse

[jira] [Commented] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646325#comment-16646325 ] Maximilian Michels commented on BEAM-5713: -- This is a "feature". See

[jira] [Assigned] (BEAM-5713) Flink portable runner schedules all tasks of streaming job on same task manager

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels reassigned BEAM-5713: Assignee: Maximilian Michels > Flink portable runner schedules all tasks of

[jira] [Commented] (BEAM-5176) FailOnWarnings behave differently between CLI and Intellij build

2018-10-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646226#comment-16646226 ] Maximilian Michels commented on BEAM-5176: -- Great! Thanks for looking into this. It appears to me

[jira] [Created] (BEAM-5687) Checkpointing in portable pipelines does not work

2018-10-09 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5687: Summary: Checkpointing in portable pipelines does not work Key: BEAM-5687 URL: https://issues.apache.org/jira/browse/BEAM-5687 Project: Beam Issue

[jira] [Issue Comment Deleted] (BEAM-5664) A canceled pipeline should not return a done status in the jobserver.

2018-10-08 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5664: - Comment: was deleted (was: Is this still being worked on?) > A canceled pipeline should

[jira] [Commented] (BEAM-3676) FlinkRunner: Portable state service

2018-10-08 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641810#comment-16641810 ] Maximilian Michels commented on BEAM-3676: -- Is this still being worked on? [~thw] We can track

[jira] [Commented] (BEAM-5664) A canceled pipeline should not return a done status in the jobserver.

2018-10-08 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641802#comment-16641802 ] Maximilian Michels commented on BEAM-5664: -- Is this still being worked on? > A canceled pipeline

[jira] [Resolved] (BEAM-5442) PortableRunner swallows custom options for Runner

2018-10-08 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5442. -- Resolution: Fixed Great! I will get in contact with you to see why the checkpointing is

[jira] [Resolved] (BEAM-5668) [beam_PostCommit_Python_Verify] [:pythonPostCommit] Many tests failing

2018-10-06 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5668. -- Resolution: Fixed Fix Version/s: Not applicable PostCommit is back to normal. >

[jira] [Commented] (BEAM-5668) [beam_PostCommit_Python_Verify] [:pythonPostCommit] Many tests failing

2018-10-06 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640685#comment-16640685 ] Maximilian Michels commented on BEAM-5668: -- Thanks [~swegner]. Yes, the error you posted is

[jira] [Commented] (BEAM-5664) A canceled pipeline should not return a done status in the jobserver.

2018-10-05 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640007#comment-16640007 ] Maximilian Michels commented on BEAM-5664: -- Cancel is not supported via the

[jira] [Resolved] (BEAM-5369) Portable wordcount java broken because of create_view usage

2018-10-05 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5369. -- Resolution: Fixed > Portable wordcount java broken because of create_view usage >

[jira] [Resolved] (BEAM-5528) Java PortableRunner pipeline fails on FlinkRunner due to CREATE_VIEW

2018-10-05 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5528. -- Resolution: Fixed > Java PortableRunner pipeline fails on FlinkRunner due to

[jira] [Commented] (BEAM-5649) Remove deprecated primitive CREATE_VIEW transform from Runner API

2018-10-05 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639559#comment-16639559 ] Maximilian Michels commented on BEAM-5649: -- Another solution that we discussed in

[jira] [Assigned] (BEAM-5649) Remove deprecated primitive CREATE_VIEW transform from Runner API

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels reassigned BEAM-5649: Assignee: (was: Maximilian Michels) > Remove deprecated primitive CREATE_VIEW

[jira] [Commented] (BEAM-5649) Remove deprecated primitive CREATE_VIEW transform from Runner API

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638639#comment-16638639 ] Maximilian Michels commented on BEAM-5649: -- The translation code of the non-portable Runners is

[jira] [Created] (BEAM-5649) Remove deprecated primitive CREATE_VIEW transform from Runner API

2018-10-04 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5649: Summary: Remove deprecated primitive CREATE_VIEW transform from Runner API Key: BEAM-5649 URL: https://issues.apache.org/jira/browse/BEAM-5649 Project: Beam

[jira] [Commented] (BEAM-4819) Make portable Flink runner JobBundleFactory configurable

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638542#comment-16638542 ] Maximilian Michels commented on BEAM-4819: -- That's what I wanted to find out. So we keep this

[jira] [Commented] (BEAM-5419) Build multiple versions of the Flink Runner against different Flink versions

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638367#comment-16638367 ] Maximilian Michels commented on BEAM-5419: -- Won't be fixed for 2.8.0. > Build multiple versions

[jira] [Commented] (BEAM-5528) Java PortableRunner pipeline fails on FlinkRunner due to CREATE_VIEW

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638379#comment-16638379 ] Maximilian Michels commented on BEAM-5528: -- Just discovered this is already reported via

[jira] [Updated] (BEAM-5369) Portable wordcount java broken because of create_view usage

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5369: - Labels: portability portability-flink (was: ) > Portable wordcount java broken because

[jira] [Updated] (BEAM-5369) Portable wordcount java broken because of create_view usage

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5369: - Fix Version/s: 2.8.0 > Portable wordcount java broken because of create_view usage >

[jira] [Updated] (BEAM-5419) Build multiple versions of the Flink Runner against different Flink versions

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5419: - Fix Version/s: (was: 2.8.0) 2.9.0 > Build multiple versions of the

[jira] [Updated] (BEAM-5267) Update Flink Runner to Flink 1.6.x

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5267: - Fix Version/s: (was: 2.8.0) 2.9.0 > Update Flink Runner to Flink

[jira] [Commented] (BEAM-5267) Update Flink Runner to Flink 1.6.x

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638366#comment-16638366 ] Maximilian Michels commented on BEAM-5267: -- This depends on BEAM-5419 and won't be fixed for

[jira] [Commented] (BEAM-4819) Make portable Flink runner JobBundleFactory configurable

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638276#comment-16638276 ] Maximilian Michels commented on BEAM-4819: -- Is this still an issue after

[jira] [Commented] (BEAM-5528) Java PortableRunner pipeline fails on FlinkRunner due to CREATE_VIEW

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638046#comment-16638046 ] Maximilian Michels commented on BEAM-5528: -- Yes, this should go into 2.8.0. Pull request is open.

[jira] [Updated] (BEAM-5528) Java PortableRunner pipeline fails on FlinkRunner due to CREATE_VIEW

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5528: - Priority: Critical (was: Major) > Java PortableRunner pipeline fails on FlinkRunner due

[jira] [Updated] (BEAM-5440) Add option to mount a directory inside SDK harness containers

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5440: - Fix Version/s: 2.9.0 > Add option to mount a directory inside SDK harness containers >

[jira] [Commented] (BEAM-5440) Add option to mount a directory inside SDK harness containers

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637980#comment-16637980 ] Maximilian Michels commented on BEAM-5440: -- [~altay] Not a blocker. I generally always provide a

[jira] [Commented] (BEAM-5442) PortableRunner swallows custom options for Runner

2018-10-04 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637961#comment-16637961 ] Maximilian Michels commented on BEAM-5442: -- Yes, this should go into 2.8.0. [~altay] You can

[jira] [Resolved] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

2018-10-03 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5187. -- Resolution: Implemented Fix Version/s: 2.7.0 Artifact staging is not integrated

[jira] [Closed] (BEAM-5527) Java PortableRunner produces wrongly shaded jar

2018-09-30 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels closed BEAM-5527. Resolution: Duplicate Fix Version/s: (was: 2.8.0) Not

[jira] [Created] (BEAM-5528) Java PortableRunner pipeline fails on FlinkRunner due to CREATE_VIEW

2018-09-30 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5528: Summary: Java PortableRunner pipeline fails on FlinkRunner due to CREATE_VIEW Key: BEAM-5528 URL: https://issues.apache.org/jira/browse/BEAM-5528 Project:

[jira] [Created] (BEAM-5527) Java PortableRunner produces wrongly shaded jar

2018-09-30 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5527: Summary: Java PortableRunner produces wrongly shaded jar Key: BEAM-5527 URL: https://issues.apache.org/jira/browse/BEAM-5527 Project: Beam Issue

[jira] [Resolved] (BEAM-5458) Python PreCommit broken

2018-09-23 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5458. -- Resolution: Fixed Assignee: Robbe Fix Version/s: Not applicable >

[jira] [Created] (BEAM-5458) Python PreCommit broken

2018-09-21 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5458: Summary: Python PreCommit broken Key: BEAM-5458 URL: https://issues.apache.org/jira/browse/BEAM-5458 Project: Beam Issue Type: Test

[jira] [Resolved] (BEAM-5444) Post Commit Failure in Flink Runner

2018-09-21 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5444. -- Resolution: Fixed Fix Version/s: Not applicable > Post Commit Failure in Flink

[jira] [Commented] (BEAM-5444) Post Commit Failure in Flink Runner

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622529#comment-16622529 ] Maximilian Michels commented on BEAM-5444: -- The default parallelism for local execution changed

[jira] [Commented] (BEAM-5442) PortableRunner swallows custom options for Runner

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622500#comment-16622500 ] Maximilian Michels commented on BEAM-5442: -- There are some pros to doing an upfront

[jira] [Commented] (BEAM-5442) PortableRunner swallows custom options for Runner

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622449#comment-16622449 ] Maximilian Michels commented on BEAM-5442: -- Actually, if you look at {{get_all_options}} method

[jira] [Assigned] (BEAM-5442) PortableRunner swallows custom options for Runner

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels reassigned BEAM-5442: Assignee: Maximilian Michels > PortableRunner swallows custom options for Runner >

[jira] [Updated] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5288: - Labels: (was: por) > Modify Environment to support non-dockerized SDK harness

[jira] [Updated] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5288: - Labels: por (was: ) > Modify Environment to support non-dockerized SDK harness

[jira] [Updated] (BEAM-5441) Portable Wordcount fails in GreedyPipelineFuser

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5441: - Labels: portability portability-flink (was: ) > Portable Wordcount fails in

[jira] [Updated] (BEAM-5440) Add option to mount a directory inside SDK harness containers

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5440: - Labels: portability portability-flink (was: ) > Add option to mount a directory inside

[jira] [Updated] (BEAM-5442) PortableRunner swallows custom options for Runner

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5442: - Labels: portability portability-flink (was: ) > PortableRunner swallows custom options

[jira] [Created] (BEAM-5442) PortableRunner swallows custom options for Runner

2018-09-20 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5442: Summary: PortableRunner swallows custom options for Runner Key: BEAM-5442 URL: https://issues.apache.org/jira/browse/BEAM-5442 Project: Beam Issue

[jira] [Created] (BEAM-5441) Portable Wordcount fails in GreedyPipelineFuser

2018-09-20 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5441: Summary: Portable Wordcount fails in GreedyPipelineFuser Key: BEAM-5441 URL: https://issues.apache.org/jira/browse/BEAM-5441 Project: Beam Issue

[jira] [Updated] (BEAM-5440) Add option to mount a directory inside SDK harness containers

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5440: - Description: While experimenting with the Python SDK locally, I found it inconvenient

[jira] [Updated] (BEAM-5440) Add option to mount a directory inside SDK harness containers

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5440: - Description: While experimenting with the Python SDK locally, I found it inconvenient

[jira] [Created] (BEAM-5440) Add option to mount a directory inside SDK harness containers

2018-09-20 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5440: Summary: Add option to mount a directory inside SDK harness containers Key: BEAM-5440 URL: https://issues.apache.org/jira/browse/BEAM-5440 Project: Beam

[jira] [Resolved] (BEAM-3089) Issue with setting the parallelism at client level using Flink runner

2018-09-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-3089. -- Resolution: Fixed Assignee: Maximilian Michels (was: Grzegorz KoĊ‚akowski) Fixed

[jira] [Resolved] (BEAM-5418) Add Flink version compatibility table to Runner page

2018-09-18 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5418. -- Resolution: Fixed Fix Version/s: 2.8.0 > Add Flink version compatibility table

[jira] [Updated] (BEAM-5385) Flink jobserver does not honor --flink-master-url

2018-09-18 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5385: - Fix Version/s: (was: 2.8.0) 2.7.0 > Flink jobserver does not honor

[jira] [Updated] (BEAM-5239) Allow configure latencyTrackingInterval

2018-09-18 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5239: - Fix Version/s: 2.7.0 > Allow configure latencyTrackingInterval >

[jira] [Updated] (BEAM-5246) Beam metrics exported as flink metrics are not correct

2018-09-18 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5246: - Fix Version/s: (was: 2.8.0) 2.7.0 > Beam metrics exported as flink

[jira] [Updated] (BEAM-5239) Allow configure latencyTrackingInterval

2018-09-18 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5239: - Fix Version/s: (was: 2.8.0) > Allow configure latencyTrackingInterval >

[jira] [Updated] (BEAM-3089) Issue with setting the parallelism at client level using Flink runner

2018-09-18 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-3089: - Fix Version/s: 2.8.0 > Issue with setting the parallelism at client level using Flink

[jira] [Commented] (BEAM-5267) Update Flink Runner to Flink 1.6.x

2018-09-18 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619100#comment-16619100 ] Maximilian Michels commented on BEAM-5267: -- We won't upgrade the main build target to 1.6 soon.

[jira] [Created] (BEAM-5419) Build multiple versions of the Flink Runner against different Flink versions

2018-09-18 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5419: Summary: Build multiple versions of the Flink Runner against different Flink versions Key: BEAM-5419 URL: https://issues.apache.org/jira/browse/BEAM-5419

[jira] [Created] (BEAM-5418) Add Flink version compatibility table to Runner page

2018-09-18 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5418: Summary: Add Flink version compatibility table to Runner page Key: BEAM-5418 URL: https://issues.apache.org/jira/browse/BEAM-5418 Project: Beam

[jira] [Commented] (BEAM-593) Support unblocking run() in FlinkRunner and cancel() and waitUntilFinish() in FlinkRunnerResult

2018-09-18 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618916#comment-16618916 ] Maximilian Michels commented on BEAM-593: - Unfortunately, this is not resolved for the non-portable

[jira] [Closed] (BEAM-5405) Remove deprecated AbstractStateBackend from FlinkPipelineOptions

2018-09-17 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels closed BEAM-5405. Resolution: Fixed 23eb70ffa5b96396f75391364d1ff37f9ce6604a > Remove deprecated

[jira] [Commented] (BEAM-5396) Flink portable runner savepoint / upgrade support

2018-09-17 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617869#comment-16617869 ] Maximilian Michels commented on BEAM-5396: -- This is not an easy one. Once the portability is

[jira] [Closed] (BEAM-5403) Update Flink Runner to 1.5.3

2018-09-17 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels closed BEAM-5403. Resolution: Fixed 79fb76b63309426820ccd0ee874c45d052db3b70 > Update Flink Runner to 1.5.3

[jira] [Created] (BEAM-5405) Remove deprecated AbstractStateBackend from FlinkPipelineOptions

2018-09-17 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5405: Summary: Remove deprecated AbstractStateBackend from FlinkPipelineOptions Key: BEAM-5405 URL: https://issues.apache.org/jira/browse/BEAM-5405 Project: Beam

[jira] [Created] (BEAM-5403) Update Flink Runner to 1.5.3

2018-09-17 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5403: Summary: Update Flink Runner to 1.5.3 Key: BEAM-5403 URL: https://issues.apache.org/jira/browse/BEAM-5403 Project: Beam Issue Type: Improvement

[jira] [Resolved] (BEAM-5337) [beam_PostCommit_Java_GradleBuild][:beam-runners-flink_2.11:test][Flake] Build times out in beam-runners-flink target

2018-09-16 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5337. -- Resolution: Fixed Fix Version/s: 2.8.0 This has been resolved with

[jira] [Resolved] (BEAM-5246) Beam metrics exported as flink metrics are not correct

2018-09-14 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5246. -- Resolution: Fixed Fix Version/s: (was: 2.7.0) 2.8.0

[jira] [Updated] (BEAM-5337) [beam_PostCommit_Java_GradleBuild][:beam-runners-flink_2.11:test][Flake] Build times out in beam-runners-flink target

2018-09-12 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5337: - Component/s: runner-flink >

[jira] [Assigned] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments

2018-09-12 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels reassigned BEAM-5288: Assignee: Ankur Goenka (was: Maximilian Michels) > Modify Environment to support

[jira] [Commented] (BEAM-5299) Define max global window as a shared value in protos like URN enums.

2018-09-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611025#comment-16611025 ] Maximilian Michels commented on BEAM-5299: -- Thanks for the pointer [~lcwik]. What I did was

[jira] [Comment Edited] (BEAM-4615) Flink job server driver wrapper

2018-09-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610895#comment-16610895 ] Maximilian Michels edited comment on BEAM-4615 at 9/11/18 4:29 PM: --- bq.

[jira] [Commented] (BEAM-4615) Flink job server driver wrapper

2018-09-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610895#comment-16610895 ] Maximilian Michels commented on BEAM-4615: -- bq. A gradle wrapper that can execute the Flink job

[jira] [Resolved] (BEAM-5239) Allow configure latencyTrackingInterval

2018-09-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5239. -- Resolution: Fixed Fix Version/s: (was: 2.7.0) 2.8.0 >

[jira] [Updated] (BEAM-5246) Beam metrics exported as flink metrics are not correct

2018-09-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5246: - Fix Version/s: 2.7.0 > Beam metrics exported as flink metrics are not correct >

[jira] [Assigned] (BEAM-5246) Beam metrics exported as flink metrics are not correct

2018-09-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels reassigned BEAM-5246: Assignee: Jozef Vilcek (was: Aljoscha Krettek) > Beam metrics exported as flink

[jira] [Updated] (BEAM-5239) Allow configure latencyTrackingInterval

2018-09-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5239: - Fix Version/s: 2.7.0 > Allow configure latencyTrackingInterval >

[jira] [Assigned] (BEAM-5239) Allow configure latencyTrackingInterval

2018-09-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels reassigned BEAM-5239: Assignee: Jozef Vilcek (was: Aljoscha Krettek) > Allow configure

[jira] [Commented] (BEAM-5299) Define max global window as a shared value in protos like URN enums.

2018-09-11 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610725#comment-16610725 ] Maximilian Michels commented on BEAM-5299: -- That is exactly the type of lookup I had in mind. I

[jira] [Commented] (BEAM-5299) Define max global window as a shared value in protos like URN enums.

2018-09-10 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16609599#comment-16609599 ] Maximilian Michels commented on BEAM-5299: -- DoFns are free to return WindowedValues with

[jira] [Assigned] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments

2018-09-07 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels reassigned BEAM-5288: Assignee: Maximilian Michels > Modify Environment to support non-dockerized SDK

[jira] [Resolved] (BEAM-5332) SDK harness containers are not eventually shut down after job ends

2018-09-07 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5332. -- Resolution: Fixed Fixed with https://github.com/apache/beam/pull/6342 > SDK harness

[jira] [Resolved] (BEAM-5331) Flink portable runner Python validate runner failures

2018-09-07 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5331. -- Resolution: Fixed Fix Version/s: 2.8.0 This is now resolved with the recent

[jira] [Created] (BEAM-5338) PortableValidatesRunner always rebuilds Python SDK Harness image

2018-09-07 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5338: Summary: PortableValidatesRunner always rebuilds Python SDK Harness image Key: BEAM-5338 URL: https://issues.apache.org/jira/browse/BEAM-5338 Project: Beam

[jira] [Resolved] (BEAM-5308) JobBundleFactory BindException with FlinkRunner and remote cluster

2018-09-06 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels resolved BEAM-5308. -- Resolution: Fixed Fix Version/s: 2.8.0 The root cause of this has been fixed,

[jira] [Created] (BEAM-5332) SDK harness containers are not eventually shut down after job ends

2018-09-06 Thread Maximilian Michels (JIRA)
Maximilian Michels created BEAM-5332: Summary: SDK harness containers are not eventually shut down after job ends Key: BEAM-5332 URL: https://issues.apache.org/jira/browse/BEAM-5332 Project: Beam

[jira] [Commented] (BEAM-5308) JobBundleFactory BindException with FlinkRunner and remote cluster

2018-09-06 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605791#comment-16605791 ] Maximilian Michels commented on BEAM-5308: -- This is a classloading issue when closing the

[jira] [Updated] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments

2018-09-06 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-5288: - Description: As of mailing discussions and BEAM-5187, it has become clear that we need to

[jira] [Commented] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments

2018-09-06 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605603#comment-16605603 ] Maximilian Michels commented on BEAM-5288: -- [~lcwik] +1 {{SupportedEnvironments}} is better.

[jira] [Assigned] (BEAM-5308) JobBundleFactory BindException with FlinkRunner and remote cluster

2018-09-05 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels reassigned BEAM-5308: Assignee: Maximilian Michels > JobBundleFactory BindException with FlinkRunner and

  1   2   >