[jira] [Resolved] (BEAM-2594) Python shim for submitting to a JobService

[ https://issues.apache.org/jira/browse/BEAM-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-2594. Resolution: Fixed Fix Version/s: 2.8.0 > Python shim for submitting to a JobService >

[jira] [Commented] (BEAM-2594) Python shim for submitting to a JobService

[ https://issues.apache.org/jira/browse/BEAM-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645738#comment-16645738 ] Ankur Goenka commented on BEAM-2594: This PR splits the ULR and make it possible to submit jobs to any

[jira] [Assigned] (BEAM-2594) Python shim for submitting to a JobService

[ https://issues.apache.org/jira/browse/BEAM-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-2594: -- Assignee: Ankur Goenka > Python shim for submitting to a JobService >

[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to pip download flake

[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645730#comment-16645730 ] Ankur Goenka commented on BEAM-5683: check_output does that by default. >

[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to pip download flake

[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645718#comment-16645718 ] Ankur Goenka commented on BEAM-5683: I agree. Shall we expose the pip logs? I think exposing when

[jira] [Created] (BEAM-5708) Support caching of SDKHarness environments in flink

Ankur Goenka created BEAM-5708: -- Summary: Support caching of SDKHarness environments in flink Key: BEAM-5708 URL: https://issues.apache.org/jira/browse/BEAM-5708 Project: Beam Issue Type:

[jira] [Resolved] (BEAM-5697) Support parsing legacy options in FnHaness

[ https://issues.apache.org/jira/browse/BEAM-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5697. Resolution: Fixed Fix Version/s: 2.8.0 > Support parsing legacy options in FnHaness >

[jira] [Created] (BEAM-5697) Support parsing legacy options in FnHaness

Ankur Goenka created BEAM-5697: -- Summary: Support parsing legacy options in FnHaness Key: BEAM-5697 URL: https://issues.apache.org/jira/browse/BEAM-5697 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642484#comment-16642484 ] Ankur Goenka commented on BEAM-5683: As mentioned earlier, the test case seems to be failing because

[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642450#comment-16642450 ] Ankur Goenka commented on BEAM-5683: Looking at it. > [beam_PostCommit_Py_VR_Dataflow]

[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job fixes

[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16640350#comment-16640350 ] Ankur Goenka commented on BEAM-5467: The test pass consistently on local jenkins setup. > Python

[jira] [Resolved] (BEAM-3904) Don't use UUID when worker_id is missing

[ https://issues.apache.org/jira/browse/BEAM-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-3904. Resolution: Fixed Fix Version/s: 2.8.0 > Don't use UUID when worker_id is missing >

[jira] [Assigned] (BEAM-4176) Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner passes

[ https://issues.apache.org/jira/browse/BEAM-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-4176: -- Assignee: Ankur Goenka > Java: Portable batch runner passes all ValidatesRunner tests that >

[jira] [Commented] (BEAM-4176) Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner passes

[ https://issues.apache.org/jira/browse/BEAM-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637390#comment-16637390 ] Ankur Goenka commented on BEAM-4176: !81VxNWtFtke.png!   We are able to run all the test cases with

[jira] [Updated] (BEAM-4176) Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner passes

[ https://issues.apache.org/jira/browse/BEAM-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-4176: --- Attachment: 81VxNWtFtke.png > Java: Portable batch runner passes all ValidatesRunner tests that >

[jira] [Resolved] (BEAM-5022) Move beam-sdks-java-fn-execution#createPortableValidatesRunnerTask to BeamModulePlugin

[ https://issues.apache.org/jira/browse/BEAM-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5022. Resolution: Fixed Fix Version/s: 2.8.0 > Move

[jira] [Updated] (BEAM-5090) Use topological sort during ProcessBundle in Java SDKHarness

[ https://issues.apache.org/jira/browse/BEAM-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-5090: --- Labels: newbie (was: ) > Use topological sort during ProcessBundle in Java SDKHarness >

[jira] [Resolved] (BEAM-4826) Flink runner sends bad flatten to SDK

[ https://issues.apache.org/jira/browse/BEAM-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-4826. Resolution: Fixed Fix Version/s: 2.8.0 > Flink runner sends bad flatten to SDK >

[jira] [Resolved] (BEAM-5194) Pipeline options with multi value are not deserialized correctly from map

[ https://issues.apache.org/jira/browse/BEAM-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5194. Resolution: Fixed Fix Version/s: 2.8.0 > Pipeline options with multi value are not

[jira] [Resolved] (BEAM-5190) Python pipeline options are not picked correctly by PortableRunner

[ https://issues.apache.org/jira/browse/BEAM-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5190. Resolution: Fixed Fix Version/s: 2.8.0 > Python pipeline options are not picked correctly by

[jira] [Commented] (BEAM-5187) Create a ProcessJobBundleFactory for non-dockerized SDK harness

[ https://issues.apache.org/jira/browse/BEAM-5187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636285#comment-16636285 ] Ankur Goenka commented on BEAM-5187: This seems to be done. Shall we close it? > Create a

[jira] [Resolved] (BEAM-2769) Java SDK support for submitting a Portable Pipeline

[ https://issues.apache.org/jira/browse/BEAM-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-2769. Resolution: Fixed Fix Version/s: 2.8.0 > Java SDK support for submitting a Portable Pipeline

[jira] [Commented] (BEAM-2769) Java SDK support for submitting a Portable Pipeline

[ https://issues.apache.org/jira/browse/BEAM-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636280#comment-16636280 ] Ankur Goenka commented on BEAM-2769: yes. We can submit portable pipelines now. > Java SDK support

[jira] [Resolved] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments

[ https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5288. Resolution: Fixed Fix Version/s: 2.8.0 > Modify Environment to support non-dockerized SDK

[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job fixes

[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631049#comment-16631049 ] Ankur Goenka commented on BEAM-5467: I verified that they get executed sequentially so that should not

[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job fixes

[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631002#comment-16631002 ] Ankur Goenka commented on BEAM-5467: Anecdotally, tasks are failing because of segfault with following

[jira] [Updated] (BEAM-5433) Cleanup Environment.url from beam_runner_api.proto

[ https://issues.apache.org/jira/browse/BEAM-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-5433: --- Issue Type: Task (was: Test) > Cleanup Environment.url from beam_runner_api.proto >

[jira] [Created] (BEAM-5433) Cleanup Environment.url from beam_runner_api.proto

Ankur Goenka created BEAM-5433: -- Summary: Cleanup Environment.url from beam_runner_api.proto Key: BEAM-5433 URL: https://issues.apache.org/jira/browse/BEAM-5433 Project: Beam Issue Type: Test

[jira] [Commented] (BEAM-5396) Flink portable runner savepoint / upgrade support

[ https://issues.apache.org/jira/browse/BEAM-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618442#comment-16618442 ] Ankur Goenka commented on BEAM-5396: Makes sense. Its a runner specific functionality so its a good

[jira] [Resolved] (BEAM-5219) Expose OutboundMessage in PubSub client

[ https://issues.apache.org/jira/browse/BEAM-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5219. Resolution: Fixed Fix Version/s: 2.8.0 > Expose OutboundMessage in PubSub client >

[jira] [Resolved] (BEAM-5262) JobState support for Reference Runner

[ https://issues.apache.org/jira/browse/BEAM-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5262. Resolution: Fixed Fix Version/s: 2.8.0 > JobState support for Reference Runner >

[jira] [Commented] (BEAM-5396) Flink portable runner savepoint / upgrade support

[ https://issues.apache.org/jira/browse/BEAM-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618330#comment-16618330 ] Ankur Goenka commented on BEAM-5396: Thanks for the reference link. Its very relevant. This feature

[jira] [Commented] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments

[ https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614240#comment-16614240 ] Ankur Goenka commented on BEAM-5288: We are thinking of removing the args as of now to avoid any

[jira] [Commented] (BEAM-5283) Enable Python Portable Flink PostCommit Tests to Jenkins

[ https://issues.apache.org/jira/browse/BEAM-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614179#comment-16614179 ] Ankur Goenka commented on BEAM-5283: Reference bug [https://github.com/pypa/virtualenv/issues/997] 

[jira] [Resolved] (BEAM-5283) Enable Python Portable Flink PostCommit Tests to Jenkins

[ https://issues.apache.org/jira/browse/BEAM-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5283. Resolution: Fixed Fix Version/s: 2.8.0 > Enable Python Portable Flink PostCommit Tests to

[jira] [Assigned] (BEAM-5283) Enable Python Portable Flink PostCommit Tests to Jenkins

[ https://issues.apache.org/jira/browse/BEAM-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-5283: -- Assignee: Ankur Goenka (was: Jason Kuster) > Enable Python Portable Flink PostCommit Tests to

[jira] [Commented] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments

[ https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611413#comment-16611413 ] Ankur Goenka commented on BEAM-5288: It will be good to expose args separately in Process Environment

[jira] [Created] (BEAM-5369) Portable wordcount java broken because of create_view usage

Ankur Goenka created BEAM-5369: -- Summary: Portable wordcount java broken because of create_view usage Key: BEAM-5369 URL: https://issues.apache.org/jira/browse/BEAM-5369 Project: Beam Issue

[jira] [Commented] (BEAM-5337) [beam_PostCommit_Java_GradleBuild][:beam-runners-flink_2.11:test][Flake] Build times out in beam-runners-flink target

[ https://issues.apache.org/jira/browse/BEAM-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607583#comment-16607583 ] Ankur Goenka commented on BEAM-5337: Tried running the failing gradle task ./gradlew

[jira] [Commented] (BEAM-5308) JobBundleFactory BindException with FlinkRunner and remote cluster

[ https://issues.apache.org/jira/browse/BEAM-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604840#comment-16604840 ] Ankur Goenka commented on BEAM-5308: I agree, the bug is setting the port range. > JobBundleFactory

[jira] [Commented] (BEAM-5283) Enable Python Portable Flink PostCommit Tests to Jenkins

[ https://issues.apache.org/jira/browse/BEAM-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604833#comment-16604833 ] Ankur Goenka commented on BEAM-5283: The root cause seems to be related to permission.    

[jira] [Created] (BEAM-5284) Enable Java Portable Flink PostCommit Tests to Jenkins

Ankur Goenka created BEAM-5284: -- Summary: Enable Java Portable Flink PostCommit Tests to Jenkins Key: BEAM-5284 URL: https://issues.apache.org/jira/browse/BEAM-5284 Project: Beam Issue Type:

[jira] [Created] (BEAM-5283) Enable Python Portable Flink PostCommit Tests to Jenkins

Ankur Goenka created BEAM-5283: -- Summary: Enable Python Portable Flink PostCommit Tests to Jenkins Key: BEAM-5283 URL: https://issues.apache.org/jira/browse/BEAM-5283 Project: Beam Issue Type:

[jira] [Created] (BEAM-5273) Local file system does not work as expected on Portability Framework with Docker

Ankur Goenka created BEAM-5273: -- Summary: Local file system does not work as expected on Portability Framework with Docker Key: BEAM-5273 URL: https://issues.apache.org/jira/browse/BEAM-5273 Project:

[jira] [Commented] (BEAM-3676) FlinkRunner: Portable state service

[ https://issues.apache.org/jira/browse/BEAM-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597133#comment-16597133 ] Ankur Goenka commented on BEAM-3676: [~axelmagn] is there any thing left on this jira?   >

[jira] [Created] (BEAM-5262) JobState support for Reference Runner

Ankur Goenka created BEAM-5262: -- Summary: JobState support for Reference Runner Key: BEAM-5262 URL: https://issues.apache.org/jira/browse/BEAM-5262 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-5219) Expose OutboundMessage in PubSub client

Ankur Goenka created BEAM-5219: -- Summary: Expose OutboundMessage in PubSub client Key: BEAM-5219 URL: https://issues.apache.org/jira/browse/BEAM-5219 Project: Beam Issue Type: Improvement

[jira] [Assigned] (BEAM-5219) Expose OutboundMessage in PubSub client

[ https://issues.apache.org/jira/browse/BEAM-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-5219: -- Assignee: Ankur Goenka (was: Chamikara Jayalath) > Expose OutboundMessage in PubSub client >

[jira] [Created] (BEAM-5218) Add the portable test name to the pipeline name for easy debugging

Ankur Goenka created BEAM-5218: -- Summary: Add the portable test name to the pipeline name for easy debugging Key: BEAM-5218 URL: https://issues.apache.org/jira/browse/BEAM-5218 Project: Beam

[jira] [Commented] (BEAM-5180) Broken FileResultCoder via parseSchema change

[ https://issues.apache.org/jira/browse/BEAM-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592727#comment-16592727 ] Ankur Goenka commented on BEAM-5180: That's correct. HDFS file URIs in this case do not have authority

[jira] [Comment Edited] (BEAM-5180) Broken FileResultCoder via parseSchema change

[ https://issues.apache.org/jira/browse/BEAM-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591973#comment-16591973 ] Ankur Goenka edited comment on BEAM-5180 at 8/25/18 10:06 PM: -- >From the 

[jira] [Commented] (BEAM-5180) Broken FileResultCoder via parseSchema change

[ https://issues.apache.org/jira/browse/BEAM-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591973#comment-16591973 ] Ankur Goenka commented on BEAM-5180: >From the java.net.URI docs, A hierarchical URI is subject to

[jira] [Commented] (BEAM-5190) Python pipeline options are not picked correctly by PortableRunner

[ https://issues.apache.org/jira/browse/BEAM-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589865#comment-16589865 ] Ankur Goenka commented on BEAM-5190: Yes, The more interesting stack is on the Python SDK side where 

[jira] [Comment Edited] (BEAM-5156) Apache Beam on dataflow runner can't find Tensorflow for workers

[ https://issues.apache.org/jira/browse/BEAM-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589137#comment-16589137 ] Ankur Goenka edited comment on BEAM-5156 at 8/22/18 5:17 PM: - Check the thread

[jira] [Commented] (BEAM-5156) Apache Beam on dataflow runner can't find Tensorflow for workers

[ https://issues.apache.org/jira/browse/BEAM-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589137#comment-16589137 ] Ankur Goenka commented on BEAM-5156: Check the thread

[jira] [Assigned] (BEAM-5194) Pipeline options with multi value are not deserialized correctly from map

[ https://issues.apache.org/jira/browse/BEAM-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-5194: -- Assignee: Ankur Goenka (was: Ahmet Altay) > Pipeline options with multi value are not

[jira] [Created] (BEAM-5194) Pipeline options with multi value are not deserialized correctly from map

Ankur Goenka created BEAM-5194: -- Summary: Pipeline options with multi value are not deserialized correctly from map Key: BEAM-5194 URL: https://issues.apache.org/jira/browse/BEAM-5194 Project: Beam

[jira] [Commented] (BEAM-5190) Python pipeline options are not picked correctly by PortableRunner

[ https://issues.apache.org/jira/browse/BEAM-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16588549#comment-16588549 ] Ankur Goenka commented on BEAM-5190: Options are rewritten to URN causing the issue.

[jira] [Commented] (BEAM-5190) Python pipeline options are not picked correctly by PortableRunner

[ https://issues.apache.org/jira/browse/BEAM-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16588483#comment-16588483 ] Ankur Goenka commented on BEAM-5190: cc: [~thw] This blocks the worker_thread option on python 

[jira] [Created] (BEAM-5190) Python pipeline options are not picked correctly by PortableRunner

Ankur Goenka created BEAM-5190: -- Summary: Python pipeline options are not picked correctly by PortableRunner Key: BEAM-5190 URL: https://issues.apache.org/jira/browse/BEAM-5190 Project: Beam

[jira] [Created] (BEAM-5167) Use concurrency information from SDK Harness in Flink Portable Runner

Ankur Goenka created BEAM-5167: -- Summary: Use concurrency information from SDK Harness in Flink Portable Runner Key: BEAM-5167 URL: https://issues.apache.org/jira/browse/BEAM-5167 Project: Beam

[jira] [Created] (BEAM-5166) Add support for upper bound and optimum concurrency in SDK

Ankur Goenka created BEAM-5166: -- Summary: Add support for upper bound and optimum concurrency in SDK Key: BEAM-5166 URL: https://issues.apache.org/jira/browse/BEAM-5166 Project: Beam Issue

[jira] [Resolved] (BEAM-5155) Custom sdk_location parameter not working with fn_api

[ https://issues.apache.org/jira/browse/BEAM-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5155. Resolution: Fixed Fix Version/s: 2.7.0 > Custom sdk_location parameter not working with

[jira] [Created] (BEAM-5155) Custom sdk_location parameter not working with fn_api

Ankur Goenka created BEAM-5155: -- Summary: Custom sdk_location parameter not working with fn_api Key: BEAM-5155 URL: https://issues.apache.org/jira/browse/BEAM-5155 Project: Beam Issue Type: Bug

[jira] [Comment Edited] (BEAM-5110) Reconile Flink JVM singleton management with deployment

[ https://issues.apache.org/jira/browse/BEAM-5110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580554#comment-16580554 ] Ankur Goenka edited comment on BEAM-5110 at 8/14/18 11:47 PM: -- With the

[jira] [Commented] (BEAM-5110) Reconile Flink JVM singleton management with deployment

[ https://issues.apache.org/jira/browse/BEAM-5110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580554#comment-16580554 ] Ankur Goenka commented on BEAM-5110: With the worker_id introduction, fn_api is capable of supporting

[jira] [Assigned] (BEAM-4130) Portable Flink runner JobService entry point in a Docker container

[ https://issues.apache.org/jira/browse/BEAM-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-4130: -- Assignee: Maximilian Michels (was: Ankur Goenka) > Portable Flink runner JobService entry

[jira] [Comment Edited] (BEAM-5110) Reconile Flink JVM singleton management with deployment

[ https://issues.apache.org/jira/browse/BEAM-5110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16578840#comment-16578840 ] Ankur Goenka edited comment on BEAM-5110 at 8/13/18 7:56 PM: - I agree. We

[jira] [Commented] (BEAM-5110) Reconile Flink JVM singleton management with deployment

[ https://issues.apache.org/jira/browse/BEAM-5110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16578840#comment-16578840 ] Ankur Goenka commented on BEAM-5110: I agree. We should provide control SDKHarness container instances

[jira] [Resolved] (BEAM-4699) BeamFileSystemArtifactServicesTest.putArtifactsSingleSmallFileTest flake

[ https://issues.apache.org/jira/browse/BEAM-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-4699. Resolution: Duplicate Fix Version/s: 2.7.0 The issue is a duplicate of BEAM-4810 Please re

[jira] [Updated] (BEAM-5090) Use topological sort during ProcessBundle in Java SDKHarness

[ https://issues.apache.org/jira/browse/BEAM-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-5090: --- Description: In reference to comment

[jira] [Updated] (BEAM-5090) Use topological sort during ProcessBundle in Java SDKHarness

[ https://issues.apache.org/jira/browse/BEAM-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-5090: --- Description: In reference to comment

[jira] [Comment Edited] (BEAM-5110) Reconile Flink JVM singleton management with deployment

[ https://issues.apache.org/jira/browse/BEAM-5110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573688#comment-16573688 ] Ankur Goenka edited comment on BEAM-5110 at 8/8/18 6:43 PM: We can invert the

[jira] [Commented] (BEAM-5110) Reconile Flink JVM singleton management with deployment

[ https://issues.apache.org/jira/browse/BEAM-5110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573688#comment-16573688 ] Ankur Goenka commented on BEAM-5110: We can invert the class loading order using 

[jira] [Created] (BEAM-5090) Use topological sort during ProcessBundle in Java SDKHarness

Ankur Goenka created BEAM-5090: -- Summary: Use topological sort during ProcessBundle in Java SDKHarness Key: BEAM-5090 URL: https://issues.apache.org/jira/browse/BEAM-5090 Project: Beam Issue

[jira] [Commented] (BEAM-4826) Flink runner sends bad flatten to SDK

[ https://issues.apache.org/jira/browse/BEAM-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566224#comment-16566224 ] Ankur Goenka commented on BEAM-4826: [~herohde] Quick question. What do you mean by "Dataflow removes

[jira] [Resolved] (BEAM-4147) Abstractions for artifact delivery via arbitrary storage backends

[ https://issues.apache.org/jira/browse/BEAM-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-4147. Resolution: Resolved Fix Version/s: 2.6.0 We have a working ArtifactStaging and

[jira] [Commented] (BEAM-3230) beam_PerformanceTests_Python is failing

[ https://issues.apache.org/jira/browse/BEAM-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566065#comment-16566065 ] Ankur Goenka commented on BEAM-3230: [~tvalentyn] The latest fail is

[jira] [Commented] (BEAM-5022) Move beam-sdks-java-fn-execution#createPortableValidatesRunnerTask to BeamModulePlugin

[ https://issues.apache.org/jira/browse/BEAM-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566055#comment-16566055 ] Ankur Goenka commented on BEAM-5022: Addressed as part of [https://github.com/apache/beam/pull/6073]

[jira] [Resolved] (BEAM-4067) Java: FlinkPortableTestRunner: runs portably via self-started local Flink

[ https://issues.apache.org/jira/browse/BEAM-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-4067. Resolution: Fixed Fix Version/s: 2.7.0 > Java: FlinkPortableTestRunner: runs portably via

[jira] [Commented] (BEAM-4067) Java: FlinkPortableTestRunner: runs portably via self-started local Flink

[ https://issues.apache.org/jira/browse/BEAM-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566051#comment-16566051 ] Ankur Goenka commented on BEAM-4067: This feature is implemented in 

[jira] [Commented] (BEAM-4826) Flink runner sends bad flatten to SDK

[ https://issues.apache.org/jira/browse/BEAM-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565838#comment-16565838 ] Ankur Goenka commented on BEAM-4826: I see.  The split of flatten is ok. I will see how we can

[jira] [Resolved] (BEAM-4748) Flaky post-commit test org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactServicesTest.putArtifactsMultipleFilesConcurrentlyTest

[ https://issues.apache.org/jira/browse/BEAM-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-4748. Resolution: Duplicate Fix Version/s: 2.7.0 > Flaky post-commit test >

[jira] [Commented] (BEAM-4748) Flaky post-commit test org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactServicesTest.putArtifactsMultipleFilesConcurrentlyTest

[ https://issues.apache.org/jira/browse/BEAM-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562415#comment-16562415 ] Ankur Goenka commented on BEAM-4748: This flaky test was fixed in

[jira] [Updated] (BEAM-5041) Java Fn SDK Harness skips unprocessed pCollections

[ https://issues.apache.org/jira/browse/BEAM-5041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-5041: --- Description: Java Sdk Harness used pCollections to keep track of computed consumers

[jira] [Updated] (BEAM-5041) Java Fn SDK Harness skips unprocessed pCollections

[ https://issues.apache.org/jira/browse/BEAM-5041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-5041: --- Description: Java Sdk Harness used pCollections to keep track of computed consumers

[jira] [Updated] (BEAM-5041) Java Fn SDK Harness skips unprocessed pCollections

[ https://issues.apache.org/jira/browse/BEAM-5041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-5041: --- Description: Java Sdk Harness used pCollections to keep track of computed consumers

[jira] [Updated] (BEAM-5041) Java Fn SDK Harness skips unprocessed pCollections

[ https://issues.apache.org/jira/browse/BEAM-5041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-5041: --- Description: Java Sdk Harness used pCollections to keep track of computed consumers

[jira] [Created] (BEAM-5041) Java Fn SDK Harness skips unprocessed pCollections

Ankur Goenka created BEAM-5041: -- Summary: Java Fn SDK Harness skips unprocessed pCollections Key: BEAM-5041 URL: https://issues.apache.org/jira/browse/BEAM-5041 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-5023) BeamFnDataGrpcClient should pass the worker_id when connecting to the RunnerHarness

Ankur Goenka created BEAM-5023: -- Summary: BeamFnDataGrpcClient should pass the worker_id when connecting to the RunnerHarness Key: BEAM-5023 URL: https://issues.apache.org/jira/browse/BEAM-5023 Project:

[jira] [Created] (BEAM-5022) Move beam-sdks-java-fn-execution#createPortableValidatesRunnerTask to BeamGradlePlugin

Ankur Goenka created BEAM-5022: -- Summary: Move beam-sdks-java-fn-execution#createPortableValidatesRunnerTask to BeamGradlePlugin Key: BEAM-5022 URL: https://issues.apache.org/jira/browse/BEAM-5022

[jira] [Assigned] (BEAM-4834) Validate cycles in org.apache.beam.runners.core.construction.graph.Network before doing topological sort

[ https://issues.apache.org/jira/browse/BEAM-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-4834: -- Assignee: Ankur Goenka (was: Kenneth Knowles) > Validate cycles in

[jira] [Created] (BEAM-4834) Validate cycles in org.apache.beam.runners.core.construction.graph.Network before doing topological sort

Ankur Goenka created BEAM-4834: -- Summary: Validate cycles in org.apache.beam.runners.core.construction.graph.Network before doing topological sort Key: BEAM-4834 URL: https://issues.apache.org/jira/browse/BEAM-4834

[jira] [Created] (BEAM-4810) Flaky test BeamFileSystemArtifactServicesTest

Ankur Goenka created BEAM-4810: -- Summary: Flaky test BeamFileSystemArtifactServicesTest Key: BEAM-4810 URL: https://issues.apache.org/jira/browse/BEAM-4810 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-4787) Ignore genrated vendored files for python container

Ankur Goenka created BEAM-4787: -- Summary: Ignore genrated vendored files for python container Key: BEAM-4787 URL: https://issues.apache.org/jira/browse/BEAM-4787 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-4784) Python SDK harness container build fails

[ https://issues.apache.org/jira/browse/BEAM-4784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543769#comment-16543769 ] Ankur Goenka commented on BEAM-4784: I am not able to reproduce it here is a working build on master

[jira] [Commented] (BEAM-4773) Flink job fails if image pull fails.

[ https://issues.apache.org/jira/browse/BEAM-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542053#comment-16542053 ] Ankur Goenka commented on BEAM-4773: Fix in PR [https://github.com/apache/beam/pull/5933] > Flink job

[jira] [Created] (BEAM-4773) Flink job fails if image pull fails.

Ankur Goenka created BEAM-4773: -- Summary: Flink job fails if image pull fails. Key: BEAM-4773 URL: https://issues.apache.org/jira/browse/BEAM-4773 Project: Beam Issue Type: Bug

[jira] [Assigned] (BEAM-4130) Portable Flink runner JobService entry point in a Docker container

[ https://issues.apache.org/jira/browse/BEAM-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-4130: -- Assignee: Ankur Goenka > Portable Flink runner JobService entry point in a Docker container >

[jira] [Resolved] (BEAM-4023) Log warning for missing worker id in FnApiControlClientPoolService

[ https://issues.apache.org/jira/browse/BEAM-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-4023. Resolution: Fixed Fix Version/s: Not applicable > Log warning for missing worker id in

  1   2   >