[jira] [Comment Edited] (BEAM-10115) Staging requirements.txt fails but staging setup.py succeeds

2020-05-28 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119112#comment-17119112 ] Ankur Goenka edited comment on BEAM-10115 at 5/28/20, 10:12 PM: An action

[jira] [Created] (BEAM-10147) Clean /tmp/dataflow-requirements-cache before downloading the requirements from requirements.txt

2020-05-28 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-10147: --- Summary: Clean /tmp/dataflow-requirements-cache before downloading the requirements from requirements.txt Key: BEAM-10147 URL: https://issues.apache.org/jira/browse/BEAM-10147

[jira] [Commented] (BEAM-10115) Staging requirements.txt fails but staging setup.py succeeds

2020-05-28 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119112#comment-17119112 ] Ankur Goenka commented on BEAM-10115: - An action item from this seems to be cleaning 

[jira] [Commented] (BEAM-10115) Staging requirements.txt fails but staging setup.py succeeds

2020-05-27 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118213#comment-17118213 ] Ankur Goenka commented on BEAM-10115: - I suspect the issue might be because of old data in

[jira] [Commented] (BEAM-9974) beam_PostRelease_NightlySnapshot failing

2020-05-26 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116999#comment-17116999 ] Ankur Goenka commented on BEAM-9974: cc: [~robertwb]  > beam_PostRelease_NightlySnapshot failing >

[jira] [Resolved] (BEAM-10060) Release new Python container image for Dataflow

2020-05-26 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-10060. - Resolution: Fixed > Release new Python container image for Dataflow >

[jira] [Updated] (BEAM-10060) Release new Python container image for Dataflow

2020-05-21 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-10060: Fix Version/s: 2.23.0 > Release new Python container image for Dataflow >

[jira] [Created] (BEAM-10060) Release new Python container image for Dataflow

2020-05-21 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-10060: --- Summary: Release new Python container image for Dataflow Key: BEAM-10060 URL: https://issues.apache.org/jira/browse/BEAM-10060 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-9707) Hardcode runner harness container image for unified worker

2020-05-20 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112559#comment-17112559 ] Ankur Goenka commented on BEAM-9707: We can delay it to 2.23 > Hardcode runner harness container

[jira] [Updated] (BEAM-9707) Hardcode runner harness container image for unified worker

2020-05-20 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9707: --- Fix Version/s: (was: 2.22.0) 2.23.0 > Hardcode runner harness container image

[jira] [Updated] (BEAM-9987) Add Gradle instruction to Java quick start guide

2020-05-13 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9987: --- Parent: BEAM-9962 Issue Type: Sub-task (was: Bug) > Add Gradle instruction to Java quick

[jira] [Updated] (BEAM-9962) CI/CD for Beam

2020-05-13 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9962: --- Issue Type: Task (was: Bug) > CI/CD for Beam > -- > > Key: BEAM-9962 >

[jira] [Created] (BEAM-9987) Add Gradle instruction to Java quick start guide

2020-05-13 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9987: -- Summary: Add Gradle instruction to Java quick start guide Key: BEAM-9987 URL: https://issues.apache.org/jira/browse/BEAM-9987 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-9962) CI/CD for Beam

2020-05-12 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9962: -- Summary: CI/CD for Beam Key: BEAM-9962 URL: https://issues.apache.org/jira/browse/BEAM-9962 Project: Beam Issue Type: Bug Components: sdk-ideas

[jira] [Resolved] (BEAM-9791) Precommit for dataflow runner v2

2020-05-11 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-9791. Resolution: Fixed > Precommit for dataflow runner v2 > > >

[jira] [Updated] (BEAM-9791) Precommit for dataflow runner v2

2020-05-11 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9791: --- Fix Version/s: 2.22.0 > Precommit for dataflow runner v2 > > >

[jira] [Created] (BEAM-9791) Precommit for dataflow runner v2

2020-04-20 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9791: -- Summary: Precommit for dataflow runner v2 Key: BEAM-9791 URL: https://issues.apache.org/jira/browse/BEAM-9791 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-9752) Too many shards in GCS

2020-04-13 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9752: -- Summary: Too many shards in GCS Key: BEAM-9752 URL: https://issues.apache.org/jira/browse/BEAM-9752 Project: Beam Issue Type: Bug Components:

[jira] [Updated] (BEAM-9735) Performance regression in Python Batch pipeline in Reshuffle

2020-04-09 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9735: --- Priority: Blocker (was: Major) > Performance regression in Python Batch pipeline in Reshuffle >

[jira] [Created] (BEAM-9735) Performance regression in Python Batch pipeline in Reshuffle

2020-04-09 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9735: -- Summary: Performance regression in Python Batch pipeline in Reshuffle Key: BEAM-9735 URL: https://issues.apache.org/jira/browse/BEAM-9735 Project: Beam Issue

[jira] [Updated] (BEAM-9707) Hardcode runner harness container image for unified worker

2020-04-08 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9707: --- Fix Version/s: (was: 2.21.0) 2.22.0 > Hardcode runner harness container image

[jira] [Commented] (BEAM-9707) Hardcode runner harness container image for unified worker

2020-04-08 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17078719#comment-17078719 ] Ankur Goenka commented on BEAM-9707: I will keep it open for cleanup.   Marking the fixed version to

[jira] [Created] (BEAM-9725) Perfomance regression in reshuffle

2020-04-08 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9725: -- Summary: Perfomance regression in reshuffle Key: BEAM-9725 URL: https://issues.apache.org/jira/browse/BEAM-9725 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-9707) Hardcode runner harness container image for unified worker

2020-04-06 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9707: -- Summary: Hardcode runner harness container image for unified worker Key: BEAM-9707 URL: https://issues.apache.org/jira/browse/BEAM-9707 Project: Beam Issue

[jira] [Commented] (BEAM-9445) Preoptimize causes error with environment setup

2020-03-30 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071400#comment-17071400 ] Ankur Goenka commented on BEAM-9445: Was trying to reproduce this error with wordcount without luck.

[jira] [Resolved] (BEAM-9287) Python Validates runner tests for Unified Worker

2020-03-30 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-9287. Resolution: Fixed > Python Validates runner tests for Unified Worker >

[jira] [Assigned] (BEAM-9445) Preoptimize causes error with environment setup

2020-03-24 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-9445: -- Assignee: Ankur Goenka > Preoptimize causes error with environment setup >

[jira] [Created] (BEAM-9504) Remove streaming test_streaming_pipeline_returns_expected_user_metrics_fnapi_it from batch validates runner

2020-03-13 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9504: -- Summary: Remove streaming test_streaming_pipeline_returns_expected_user_metrics_fnapi_it from batch validates runner Key: BEAM-9504 URL:

[jira] [Comment Edited] (BEAM-9499) test_multi_triggered_gbk_side_input is failing on head

2020-03-13 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058545#comment-17058545 ] Ankur Goenka edited comment on BEAM-9499 at 3/13/20, 9:22 AM: -- [~ruoyun] I

[jira] [Commented] (BEAM-9499) test_multi_triggered_gbk_side_input is failing on head

2020-03-13 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058545#comment-17058545 ] Ankur Goenka commented on BEAM-9499: [~ruoyun] Can you take a look.  Please feel free to re-assign it

[jira] [Assigned] (BEAM-9499) test_multi_triggered_gbk_side_input is failing on head

2020-03-13 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-9499: -- Assignee: Ruoyun Huang > test_multi_triggered_gbk_side_input is failing on head >

[jira] [Commented] (BEAM-9499) test_multi_triggered_gbk_side_input is failing on head

2020-03-13 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058540#comment-17058540 ] Ankur Goenka commented on BEAM-9499: Sickbaying this test as this test has never truly run on

[jira] [Updated] (BEAM-9499) test_multi_triggered_gbk_side_input is failing on head

2020-03-13 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9499: --- Description: test_multi_triggered_gbk_side_input is failing after it was fixed to run on Dataflow

[jira] [Created] (BEAM-9499) test_multi_triggered_gbk_side_input is failing on head

2020-03-13 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9499: -- Summary: test_multi_triggered_gbk_side_input is failing on head Key: BEAM-9499 URL: https://issues.apache.org/jira/browse/BEAM-9499 Project: Beam Issue Type:

[jira] [Reopened] (BEAM-9287) Python Validates runner tests for Unified Worker

2020-03-12 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reopened BEAM-9287: We need to disable the test for batch validates runner as TeasStreams is only supported in streaming

[jira] [Commented] (BEAM-9485) Dataflow Silently drops Non implemented transform in fnapi mode.

2020-03-12 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058316#comment-17058316 ] Ankur Goenka commented on BEAM-9485: Thanks for taking a look. Happy to answer.  Is this a regression

[jira] [Commented] (BEAM-9485) Dataflow Silently drops Non implemented transform in fnapi mode.

2020-03-12 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058159#comment-17058159 ] Ankur Goenka commented on BEAM-9485: [~ruiwang] We should also cherry pick this change as it silently

[jira] [Assigned] (BEAM-9485) Dataflow Silently drops Non implemented transform in fnapi mode.

2020-03-12 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-9485: -- Assignee: Ankur Goenka > Dataflow Silently drops Non implemented transform in fnapi mode. >

[jira] [Updated] (BEAM-9485) Dataflow Silently drops Non implemented transform in fnapi mode.

2020-03-12 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9485: --- Fix Version/s: 2.20.0 > Dataflow Silently drops Non implemented transform in fnapi mode. >

[jira] [Created] (BEAM-9485) Dataflow Silently drops Non implemented transform in fnapi mode.

2020-03-11 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9485: -- Summary: Dataflow Silently drops Non implemented transform in fnapi mode. Key: BEAM-9485 URL: https://issues.apache.org/jira/browse/BEAM-9485 Project: Beam

[jira] [Assigned] (BEAM-9465) Reshuffle should trigger repeatedly

2020-03-06 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-9465: -- Assignee: Ankur Goenka > Reshuffle should trigger repeatedly >

[jira] [Created] (BEAM-9465) Reshuffle should trigger repeatedly

2020-03-06 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9465: -- Summary: Reshuffle should trigger repeatedly Key: BEAM-9465 URL: https://issues.apache.org/jira/browse/BEAM-9465 Project: Beam Issue Type: Bug

[jira] [Updated] (BEAM-9431) test_streaming_pipeline_returns_expected_user_metrics_fnapi_it is not supported with streaming engine

2020-03-03 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9431: --- Component/s: runner-dataflow > test_streaming_pipeline_returns_expected_user_metrics_fnapi_it is not

[jira] [Updated] (BEAM-9431) test_streaming_pipeline_returns_expected_user_metrics_fnapi_it is not supported with streaming engine

2020-03-03 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9431: --- Description: test_streaming_pipeline_returns_expected_user_metrics_fnapi_it is failing on Dataflow V2

[jira] [Created] (BEAM-9431) test_streaming_pipeline_returns_expected_user_metrics_fnapi_it is not supported with streaming engine

2020-03-03 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9431: -- Summary: test_streaming_pipeline_returns_expected_user_metrics_fnapi_it is not supported with streaming engine Key: BEAM-9431 URL: https://issues.apache.org/jira/browse/BEAM-9431

[jira] [Updated] (BEAM-9287) Python Validates runner tests for Unified Worker

2020-03-02 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9287: --- Fix Version/s: (was: 2.20.0) 2.21.0 > Python Validates runner tests for

[jira] [Resolved] (BEAM-9287) Python Validates runner tests for Unified Worker

2020-03-02 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-9287. Resolution: Fixed > Python Validates runner tests for Unified Worker >

[jira] [Commented] (BEAM-9287) Python Validates runner tests for Unified Worker

2020-03-02 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049629#comment-17049629 ] Ankur Goenka commented on BEAM-9287: The tests are enabled now. > Python Validates runner tests for

[jira] [Reopened] (BEAM-9287) Python Validates runner tests for Unified Worker

2020-02-27 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reopened BEAM-9287: > Python Validates runner tests for Unified Worker > >

[jira] [Updated] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner

2020-02-27 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9402: --- Description: The option provided in the test overwrite the argument provided during validates runner

[jira] [Created] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner

2020-02-27 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9402: -- Summary: test_multi_triggered_gbk_side_input is always using DirectRunner Key: BEAM-9402 URL: https://issues.apache.org/jira/browse/BEAM-9402 Project: Beam

[jira] [Created] (BEAM-9391) Cleanup hardcoded unified worker images

2020-02-26 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9391: -- Summary: Cleanup hardcoded unified worker images Key: BEAM-9391 URL: https://issues.apache.org/jira/browse/BEAM-9391 Project: Beam Issue Type: Bug

[jira] [Resolved] (BEAM-9347) Remove default image for Unified Worker

2020-02-24 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-9347. Resolution: Fixed > Remove default image for Unified Worker >

[jira] [Commented] (BEAM-9347) Remove default image for Unified Worker

2020-02-24 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043982#comment-17043982 ] Ankur Goenka commented on BEAM-9347: Yes, The PR is merged.  > Remove default image for Unified

[jira] [Updated] (BEAM-9347) Remove default image for Unified Worker

2020-02-20 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9347: --- Description: The runner will choose the Runner Harness image for UW so we don't need to overwrite

[jira] [Updated] (BEAM-9347) Remove default image for Unified Worker

2020-02-20 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9347: --- Component/s: (was: testing) > Remove default image for Unified Worker >

[jira] [Updated] (BEAM-9347) Remove default image for Unified Worker

2020-02-20 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9347: --- Fix Version/s: 2.20.0 > Remove default image for Unified Worker >

[jira] [Created] (BEAM-9347) Remove default image for Unified Worker

2020-02-20 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9347: -- Summary: Remove default image for Unified Worker Key: BEAM-9347 URL: https://issues.apache.org/jira/browse/BEAM-9347 Project: Beam Issue Type: Test

[jira] [Resolved] (BEAM-9290) runner_harness_container_image experiment is not honored in python released sdks.

2020-02-20 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-9290. Resolution: Fixed > runner_harness_container_image experiment is not honored in python released >

[jira] [Commented] (BEAM-4032) Support staging binary distributions of dependency packages.

2020-02-19 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-4032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040489#comment-17040489 ] Ankur Goenka commented on BEAM-4032: This will become less of a concern when we start supporting

[jira] [Commented] (BEAM-9299) Upgrade Flink Runner to 1.8.3 and 1.9.2

2020-02-14 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037271#comment-17037271 ] Ankur Goenka commented on BEAM-9299: +1 for option 2.  We should not make deprecation of old flink

[jira] [Updated] (BEAM-9290) runner_harness_container_image experiment is not honored in python released sdks.

2020-02-10 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9290: --- Fix Version/s: 2.20.0 > runner_harness_container_image experiment is not honored in python released

[jira] [Assigned] (BEAM-9290) runner_harness_container_image experiment is not honored in python released sdks.

2020-02-10 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-9290: -- Assignee: Ankur Goenka > runner_harness_container_image experiment is not honored in python

[jira] [Created] (BEAM-9290) runner_harness_container_image experiment is not honored in python released sdks.

2020-02-10 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9290: -- Summary: runner_harness_container_image experiment is not honored in python released sdks. Key: BEAM-9290 URL: https://issues.apache.org/jira/browse/BEAM-9290 Project:

[jira] [Updated] (BEAM-9290) runner_harness_container_image experiment is not honored in python released sdks.

2020-02-10 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9290: --- Issue Type: Bug (was: Test) > runner_harness_container_image experiment is not honored in python

[jira] [Created] (BEAM-9287) Python Validates runner tests for Unified Worker

2020-02-10 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9287: -- Summary: Python Validates runner tests for Unified Worker Key: BEAM-9287 URL: https://issues.apache.org/jira/browse/BEAM-9287 Project: Beam Issue Type: Test

[jira] [Created] (BEAM-9220) Add use_runner_v2 argument for dataflow

2020-01-29 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9220: -- Summary: Add use_runner_v2 argument for dataflow Key: BEAM-9220 URL: https://issues.apache.org/jira/browse/BEAM-9220 Project: Beam Issue Type: New Feature

[jira] [Commented] (BEAM-8970) Spark portable runner supports Yarn

2020-01-27 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024818#comment-17024818 ] Ankur Goenka commented on BEAM-8970: {code:java} ['--runner=SparkRunner',

[jira] [Created] (BEAM-9093) Pipeline options which with different underlying store variable does not get over written

2020-01-11 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9093: -- Summary: Pipeline options which with different underlying store variable does not get over written Key: BEAM-9093 URL: https://issues.apache.org/jira/browse/BEAM-9093

[jira] [Updated] (BEAM-9002) test_flatten_same_pcollections (apache_beam.transforms.ptransform_test.PTransformTest) does not work in Streaming VR suite on Dataflow

2020-01-09 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-9002: --- Component/s: (was: sdk-py-core) runner-dataflow > test_flatten_same_pcollections

[jira] [Commented] (BEAM-9002) test_flatten_same_pcollections (apache_beam.transforms.ptransform_test.PTransformTest) does not work in Streaming VR suite on Dataflow

2020-01-09 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012263#comment-17012263 ] Ankur Goenka commented on BEAM-9002: I would consider it a bug as it works on direct runner. The

[jira] [Created] (BEAM-9081) Fail Dataflow Streaming pipeline on passert failure

2020-01-09 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-9081: -- Summary: Fail Dataflow Streaming pipeline on passert failure Key: BEAM-9081 URL: https://issues.apache.org/jira/browse/BEAM-9081 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-9002) test_flatten_same_pcollections (apache_beam.transforms.ptransform_test.PTransformTest) does not work in Streaming VR suite on Dataflow

2020-01-09 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012243#comment-17012243 ] Ankur Goenka commented on BEAM-9002: Based on this error stack, same Pcollection flatten is not

[jira] [Commented] (BEAM-9002) test_flatten_same_pcollections (apache_beam.transforms.ptransform_test.PTransformTest) does not work in Streaming VR suite on Dataflow

2020-01-09 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012242#comment-17012242 ] Ankur Goenka commented on BEAM-9002: Dataflow streaming VR tests are actually timing out in case of

[jira] [Commented] (BEAM-9002) test_flatten_same_pcollections (apache_beam.transforms.ptransform_test.PTransformTest) does not work in Streaming VR suite on Dataflow

2020-01-09 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012203#comment-17012203 ] Ankur Goenka commented on BEAM-9002: The test should work fine unless there is a bug. I think timeout

[jira] [Comment Edited] (BEAM-9002) test_flatten_same_pcollections (apache_beam.transforms.ptransform_test.PTransformTest) does not work in Streaming VR suite on Dataflow

2020-01-09 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012203#comment-17012203 ] Ankur Goenka edited comment on BEAM-9002 at 1/9/20 8:23 PM: The test should

[jira] [Commented] (BEAM-8413) test_streaming_pipeline_returns_expected_user_metrics_fnapi_it failed on latest PostCommit Py36

2020-01-07 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010224#comment-17010224 ] Ankur Goenka commented on BEAM-8413: The latest runs are passing

[jira] [Updated] (BEAM-8945) DirectStreamObserver race condition

2020-01-07 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-8945: --- Description: The DirectStreamObserver can get into a dead lock if the channel become unhealthy of is

[jira] [Updated] (BEAM-6499) Support HDFS for artifact staging with Flink Runner

2020-01-07 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka updated BEAM-6499: --- Issue Type: New Feature (was: Bug) > Support HDFS for artifact staging with Flink Runner >

[jira] [Commented] (BEAM-6499) Support HDFS for artifact staging with Flink Runner

2020-01-07 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010221#comment-17010221 ] Ankur Goenka commented on BEAM-6499: We have introduced uber jar mode of pipeline submission which

[jira] [Resolved] (BEAM-5156) Apache Beam on dataflow runner can't find Tensorflow for workers

2020-01-07 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-5156. Fix Version/s: (was: 2.6.0) (was: 2.5.0) 2.17.0

[jira] [Commented] (BEAM-5156) Apache Beam on dataflow runner can't find Tensorflow for workers

2020-01-07 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010220#comment-17010220 ] Ankur Goenka commented on BEAM-5156: We have moved to an unbounded thread factory and this issue does

[jira] [Resolved] (BEAM-7243) Release python{36,37}-fnapi containers images required by Dataflow runner.

2020-01-07 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka resolved BEAM-7243. Fix Version/s: 2.16.0 Resolution: Fixed > Release python{36,37}-fnapi containers images

[jira] [Commented] (BEAM-7243) Release python{36,37}-fnapi containers images required by Dataflow runner.

2020-01-07 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010217#comment-17010217 ] Ankur Goenka commented on BEAM-7243: The PR was committed and the latest container is available for

[jira] [Commented] (BEAM-9046) Kafka connector for Python throws ClassCastException when reading KafkaRecord

2020-01-07 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010192#comment-17010192 ] Ankur Goenka commented on BEAM-9046: cc: [~mxm] [~chamikara] > Kafka connector for Python throws

[jira] [Commented] (BEAM-8980) Running GroupByKeyLoadTest on Portable Flink fails

2019-12-17 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998487#comment-16998487 ] Ankur Goenka commented on BEAM-8980: The stack trace is generally not the root cause as it is printed

[jira] [Created] (BEAM-8945) DirectStreamObserver race condition

2019-12-10 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-8945: -- Summary: DirectStreamObserver race condition Key: BEAM-8945 URL: https://issues.apache.org/jira/browse/BEAM-8945 Project: Beam Issue Type: Bug

[jira] [Comment Edited] (BEAM-6499) Support HDFS for artifact staging with Flink Runner

2019-12-06 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990239#comment-16990239 ] Ankur Goenka edited comment on BEAM-6499 at 12/7/19 12:09 AM: -- We were not

[jira] [Commented] (BEAM-6499) Support HDFS for artifact staging with Flink Runner

2019-12-06 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990239#comment-16990239 ] Ankur Goenka commented on BEAM-6499: We were not able to prioritize it yet and don't have a ETA yet.

[jira] [Created] (BEAM-8479) Python pattern for side input

2019-10-24 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-8479: -- Summary: Python pattern for side input Key: BEAM-8479 URL: https://issues.apache.org/jira/browse/BEAM-8479 Project: Beam Issue Type: Bug Components:

[jira] [Created] (BEAM-8477) Prioritize precommit over post commit jobs

2019-10-24 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-8477: -- Summary: Prioritize precommit over post commit jobs Key: BEAM-8477 URL: https://issues.apache.org/jira/browse/BEAM-8477 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-8476) Flag to skip triggering precommit jobs

2019-10-24 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-8476: -- Summary: Flag to skip triggering precommit jobs Key: BEAM-8476 URL: https://issues.apache.org/jira/browse/BEAM-8476 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-8475) Kill old jenkins job for a PR when PR is updated

2019-10-24 Thread Ankur Goenka (Jira)
Ankur Goenka created BEAM-8475: -- Summary: Kill old jenkins job for a PR when PR is updated Key: BEAM-8475 URL: https://issues.apache.org/jira/browse/BEAM-8475 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-8183) Optionally bundle multiple pipelines into a single Flink jar

2019-10-02 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943201#comment-16943201 ] Ankur Goenka commented on BEAM-8183: I agree, changing a bit of configuration in the proto will serve

[jira] [Commented] (BEAM-8183) Optionally bundle multiple pipelines into a single Flink jar

2019-10-02 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943161#comment-16943161 ] Ankur Goenka commented on BEAM-8183: I see. Thanks for explaining the use case. I think hardcoded

[jira] [Commented] (BEAM-8183) Optionally bundle multiple pipelines into a single Flink jar

2019-09-30 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941330#comment-16941330 ] Ankur Goenka commented on BEAM-8183: Flink has some neat feature of picking the pipeline on the fly. 

[jira] [Assigned] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-27 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-8314: -- Assignee: Ankur Goenka > Beam Fn Api metrics piling causes pipeline to stuck after running for

[jira] [Assigned] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-27 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka reassigned BEAM-8314: -- Assignee: (was: Ankur Goenka) > Beam Fn Api metrics piling causes pipeline to stuck after

[jira] [Commented] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-27 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939820#comment-16939820 ] Ankur Goenka commented on BEAM-8314: Cherry pick is out [https://github.com/apache/beam/pull/9682] >

[jira] [Commented] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-27 Thread Ankur Goenka (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939758#comment-16939758 ] Ankur Goenka commented on BEAM-8314: We are refining the PR to make make it more readable and smaller.

  1   2   3   4   >