[jira] [Assigned] (BEAM-5148) Implement MongoDB IO for Python SDK

2019-06-06 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-5148: - Assignee: Yichi Zhang (was: Pascal Gula) > Implement MongoDB IO for Python SDK >

[jira] [Commented] (BEAM-5148) Implement MongoDB IO for Python SDK

2019-06-06 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858169#comment-16858169 ] Yichi Zhang commented on BEAM-5148: --- Hi, [~GeoloeG_IsT], haven't heard back from you so I'm gonna

[jira] [Issue Comment Deleted] (BEAM-6696) GroupIntoBatches transform for Python SDK

2019-06-03 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-6696: -- Comment: was deleted (was: Hi, [~shehzaadn] are you actively working on this task? Do you mind if I

[jira] [Commented] (BEAM-6696) GroupIntoBatches transform for Python SDK

2019-06-03 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855203#comment-16855203 ] Yichi Zhang commented on BEAM-6696: --- Hi, [~shehzaadn] are you actively working on this task? Do you mind

[jira] [Created] (BEAM-7586) Add Integration Test for MongoDbIO in python sdk

2019-06-18 Thread Yichi Zhang (JIRA)
Yichi Zhang created BEAM-7586: - Summary: Add Integration Test for MongoDbIO in python sdk Key: BEAM-7586 URL: https://issues.apache.org/jira/browse/BEAM-7586 Project: Beam Issue Type: Sub-task

[jira] [Updated] (BEAM-7586) Add Integration Test for MongoDbIO in python sdk

2019-06-18 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-7586: -- Status: Open (was: Triage Needed) > Add Integration Test for MongoDbIO in python sdk >

[jira] [Work started] (BEAM-7586) Add Integration Test for MongoDbIO in python sdk

2019-06-18 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on BEAM-7586 started by Yichi Zhang. - > Add Integration Test for MongoDbIO in python sdk >

[jira] [Closed] (BEAM-7618) Revert moving mongodbio import to try block

2019-06-24 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-7618. - Resolution: Invalid Fix Version/s: Not applicable > Revert moving mongodbio import to try block >

[jira] [Created] (BEAM-7618) Revert moving mongodbio import to try block

2019-06-21 Thread Yichi Zhang (JIRA)
Yichi Zhang created BEAM-7618: - Summary: Revert moving mongodbio import to try block Key: BEAM-7618 URL: https://issues.apache.org/jira/browse/BEAM-7618 Project: Beam Issue Type: Improvement

[jira] [Resolved] (BEAM-7586) Add Integration Test for MongoDbIO in python sdk

2019-06-29 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-7586. --- Resolution: Fixed Fix Version/s: 2.15.0 > Add Integration Test for MongoDbIO in python sdk >

[jira] [Resolved] (BEAM-5148) Implement MongoDB IO for Python SDK

2019-06-29 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-5148. --- Resolution: Fixed Fix Version/s: (was: Not applicable) 2.14.0 >

[jira] [Work started] (BEAM-7642) (Dataflow) Python AfterProcessingTime fires before defined time

2019-08-21 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on BEAM-7642 started by Yichi Zhang. - > (Dataflow) Python AfterProcessingTime fires before defined time >

[jira] [Assigned] (BEAM-7642) (Dataflow) Python AfterProcessingTime fires before defined time

2019-08-21 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-7642: - Assignee: Yichi Zhang > (Dataflow) Python AfterProcessingTime fires before defined time >

[jira] [Resolved] (BEAM-7642) (Dataflow) Python AfterProcessingTime fires before defined time

2019-08-23 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-7642. --- Resolution: Fixed > (Dataflow) Python AfterProcessingTime fires before defined time >

[jira] [Resolved] (BEAM-7804) Fix unclear python programming guide document

2019-08-23 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-7804. --- Fix Version/s: 2.16.0 Resolution: Fixed > Fix unclear python programming guide document >

[jira] [Updated] (BEAM-6723) Reshuffle in streaming pipeline does not work on dataflowrunner python

2019-09-17 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-6723: -- Fix Version/s: 2.16.0 > Reshuffle in streaming pipeline does not work on dataflowrunner python >

[jira] [Resolved] (BEAM-6723) Reshuffle in streaming pipeline does not work on dataflowrunner python

2019-09-17 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-6723. --- Resolution: Fixed > Reshuffle in streaming pipeline does not work on dataflowrunner python >

[jira] [Commented] (BEAM-7866) Python MongoDB IO performance and correctness issues

2019-07-31 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897522#comment-16897522 ] Yichi Zhang commented on BEAM-7866: --- Thanks for looking into this, I'll try to address them asap. >

[jira] [Commented] (BEAM-7866) Python MongoDB IO performance and correctness issues

2019-08-06 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901247#comment-16901247 ] Yichi Zhang commented on BEAM-7866: --- PR is under review, hopefully I can get this done this week. >

[jira] [Commented] (BEAM-7866) Python MongoDB IO performance and correctness issues

2019-08-01 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898231#comment-16898231 ] Yichi Zhang commented on BEAM-7866: --- I looked into the split key methods using splitVector in java

[jira] [Resolved] (BEAM-7761) Support transform_name_mapping option in python sdk

2019-07-22 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-7761. --- Resolution: Fixed > Support transform_name_mapping option in python sdk >

[jira] [Closed] (BEAM-7586) Add Integration Test for MongoDbIO in python sdk

2019-07-22 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-7586. - > Add Integration Test for MongoDbIO in python sdk > - > >

[jira] [Created] (BEAM-7761) Support transform_name_mapping option in python sdk

2019-07-17 Thread Yichi Zhang (JIRA)
Yichi Zhang created BEAM-7761: - Summary: Support transform_name_mapping option in python sdk Key: BEAM-7761 URL: https://issues.apache.org/jira/browse/BEAM-7761 Project: Beam Issue Type:

[jira] [Updated] (BEAM-7761) Support transform_name_mapping option in python sdk

2019-07-17 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-7761: -- Status: Open (was: Triage Needed) > Support transform_name_mapping option in python sdk >

[jira] [Updated] (BEAM-7804) Fix unclear python programming guide document

2019-07-23 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-7804: -- Summary: Fix unclear python programming guide document (was: Fix unclear python programming guide

[jira] [Created] (BEAM-7804) Fix unclear python programming guide document on https://beam.apache.org/documentation/programming-guide/#additional-outputs

2019-07-23 Thread Yichi Zhang (JIRA)
Yichi Zhang created BEAM-7804: - Summary: Fix unclear python programming guide document on https://beam.apache.org/documentation/programming-guide/#additional-outputs Key: BEAM-7804 URL:

[jira] [Assigned] (BEAM-7804) Fix unclear python programming guide document

2019-07-23 Thread Yichi Zhang (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-7804: - Assignee: Yichi Zhang > Fix unclear python programming guide document >

[jira] [Updated] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-26 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8314: -- Attachment: E4UaSUhJJKF.png > Beam Fn Api metrics piling causes pipeline to stuck after running for a

[jira] [Updated] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-26 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8314: -- Description: Seems that in StreamingDataflowWorker we are not able to update the metrics fast enough

[jira] [Updated] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-26 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8314: -- Description: Seems that in StreamingDataflowWorker we are not able to update the metrics fast enough

[jira] [Updated] (BEAM-8320) Dataflow runner metric glitch for python streaming

2019-09-27 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8320: -- Component/s: sdk-py-core > Dataflow runner metric glitch for python streaming >

[jira] [Updated] (BEAM-8320) Dataflow runner metric glitch for python streaming

2019-09-27 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8320: -- Attachment: metric_glitch.png > Dataflow runner metric glitch for python streaming >

[jira] [Updated] (BEAM-8320) Dataflow runner metric glitch for python streaming

2019-09-27 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8320: -- Attachment: (was: metric_glitch.png) > Dataflow runner metric glitch for python streaming >

[jira] [Updated] (BEAM-8320) Dataflow runner metric glitch for python streaming

2019-09-27 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8320: -- Description: After running a python streaming pipeline for a while, sometimes the user defined metric

[jira] [Updated] (BEAM-8320) Dataflow runner metric glitch for python streaming

2019-09-27 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8320: -- Attachment: metric_glitch.png > Dataflow runner metric glitch for python streaming >

[jira] [Created] (BEAM-8320) Dataflow runner metric glitch for python streaming

2019-09-27 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8320: - Summary: Dataflow runner metric glitch for python streaming Key: BEAM-8320 URL: https://issues.apache.org/jira/browse/BEAM-8320 Project: Beam Issue Type: Bug

[jira] [Resolved] (BEAM-7045) Element counters in the Web UI graph representations for transforms for Python streaming jobs in Google Cloud Dataflow

2019-09-23 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-7045. --- Resolution: Fixed > Element counters in the Web UI graph representations for transforms for > Python

[jira] [Updated] (BEAM-7045) Element counters in the Web UI graph representations for transforms for Python streaming jobs in Google Cloud Dataflow

2019-09-23 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-7045: -- Fix Version/s: 2.16.0 > Element counters in the Web UI graph representations for transforms for >

[jira] [Closed] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-30 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-8314. - Resolution: Fixed > Beam Fn Api metrics piling causes pipeline to stuck after running for a while >

[jira] [Work started] (BEAM-7919) Add a Python 3 test scenario for MongoDB IO

2019-09-30 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on BEAM-7919 started by Yichi Zhang. - > Add a Python 3 test scenario for MongoDB IO >

[jira] [Resolved] (BEAM-7919) Add a Python 3 test scenario for MongoDB IO

2019-09-30 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-7919. --- Fix Version/s: 2.17.0 Resolution: Fixed > Add a Python 3 test scenario for MongoDB IO >

[jira] [Updated] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-25 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8314: -- Description: Seems that in StreamingDataflowWorker we are not able to update the metrics fast enough

[jira] [Commented] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-25 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938187#comment-16938187 ] Yichi Zhang commented on BEAM-8314: --- CC: [~markflyhigh] [~angoenka] > Beam Fn Api metrics piling causes

[jira] [Created] (BEAM-8314) Beam Fn Api metrics piling causes pipeline to stuck after running for a while

2019-09-25 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8314: - Summary: Beam Fn Api metrics piling causes pipeline to stuck after running for a while Key: BEAM-8314 URL: https://issues.apache.org/jira/browse/BEAM-8314 Project: Beam

[jira] [Updated] (BEAM-8652) test_streaming_pipeline_returns_expected_user_metrics_fnapi_it test flaky

2019-11-13 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8652: -- Attachment: BJL2pfOB2Qa.png > test_streaming_pipeline_returns_expected_user_metrics_fnapi_it test flaky

[jira] [Commented] (BEAM-8652) test_streaming_pipeline_returns_expected_user_metrics_fnapi_it test flaky

2019-11-13 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973725#comment-16973725 ] Yichi Zhang commented on BEAM-8652: --- It seems to be related to

[jira] [Commented] (BEAM-8652) test_streaming_pipeline_returns_expected_user_metrics_fnapi_it test flaky

2019-11-14 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974657#comment-16974657 ] Yichi Zhang commented on BEAM-8652: --- CC: [~tvalentyn] >

[jira] [Updated] (BEAM-8626) Implement status api handler in python sdk harness

2019-11-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8626: -- Status: Open (was: Triage Needed) > Implement status api handler in python sdk harness >

[jira] [Assigned] (BEAM-8626) Implement status api handler in python sdk harness

2019-11-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-8626: - Assignee: Yichi Zhang > Implement status api handler in python sdk harness >

[jira] [Created] (BEAM-8626) Implement status api handler in python sdk harness

2019-11-12 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8626: - Summary: Implement status api handler in python sdk harness Key: BEAM-8626 URL: https://issues.apache.org/jira/browse/BEAM-8626 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-8625) Implement servlet in Dataflow runner for sdk status query endpoint

2019-11-12 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8625: - Summary: Implement servlet in Dataflow runner for sdk status query endpoint Key: BEAM-8625 URL: https://issues.apache.org/jira/browse/BEAM-8625 Project: Beam

[jira] [Created] (BEAM-8624) Implement FnService for status api in Dataflow runner

2019-11-12 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8624: - Summary: Implement FnService for status api in Dataflow runner Key: BEAM-8624 URL: https://issues.apache.org/jira/browse/BEAM-8624 Project: Beam Issue Type:

[jira] [Updated] (BEAM-8614) Expose SDK harness status to Runner through FnApi

2019-11-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8614: -- Status: Open (was: Triage Needed) > Expose SDK harness status to Runner through FnApi >

[jira] [Created] (BEAM-8623) Add additional message field to Provision API response for passing status endpoint

2019-11-12 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8623: - Summary: Add additional message field to Provision API response for passing status endpoint Key: BEAM-8623 URL: https://issues.apache.org/jira/browse/BEAM-8623 Project:

[jira] [Assigned] (BEAM-8623) Add additional message field to Provision API response for passing status endpoint

2019-11-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-8623: - Assignee: Yichi Zhang > Add additional message field to Provision API response for passing

[jira] [Updated] (BEAM-8623) Add additional message field to Provision API response for passing status endpoint

2019-11-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8623: -- Status: Open (was: Triage Needed) > Add additional message field to Provision API response for passing

[jira] [Created] (BEAM-8614) Expose SDK harness status to Runner through FnApi

2019-11-11 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8614: - Summary: Expose SDK harness status to Runner through FnApi Key: BEAM-8614 URL: https://issues.apache.org/jira/browse/BEAM-8614 Project: Beam Issue Type: New

[jira] [Assigned] (BEAM-8614) Expose SDK harness status to Runner through FnApi

2019-11-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-8614: - Assignee: Yichi Zhang > Expose SDK harness status to Runner through FnApi >

[jira] [Updated] (BEAM-8624) Implement FnService for status api in Dataflow runner

2019-11-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8624: -- Status: Open (was: Triage Needed) > Implement FnService for status api in Dataflow runner >

[jira] [Created] (BEAM-8736) Support window allowed lateness in python sdk

2019-11-18 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8736: - Summary: Support window allowed lateness in python sdk Key: BEAM-8736 URL: https://issues.apache.org/jira/browse/BEAM-8736 Project: Beam Issue Type: New Feature

[jira] [Assigned] (BEAM-8580) Request Python API to support windows ClosingBehavior

2019-12-09 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-8580: - Assignee: Yichi Zhang > Request Python API to support windows ClosingBehavior >

[jira] [Comment Edited] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993141#comment-16993141 ] Yichi Zhang edited comment on BEAM-8944 at 12/11/19 2:28 AM: - CC: [~angoenka]

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993141#comment-16993141 ] Yichi Zhang commented on BEAM-8944: --- CC: [~angoenka] > Python SDK harness performance degradation with

[jira] [Created] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8944: - Summary: Python SDK harness performance degradation with UnboundedThreadPoolExecutor Key: BEAM-8944 URL: https://issues.apache.org/jira/browse/BEAM-8944 Project: Beam

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Resolved] (BEAM-8886) Add a python mongodbio integration test that triggers load split

2019-12-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8886. --- Fix Version/s: 2.19.0 Resolution: Fixed > Add a python mongodbio integration test that

[jira] [Resolved] (BEAM-8884) Python MongoDBIO TypeError when splitting

2019-12-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8884. --- Resolution: Fixed > Python MongoDBIO TypeError when splitting >

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Affects Version/s: (was: 2.17.0) > Python SDK harness performance degradation with

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Attachment: profiling_twelve_threads.png > Python SDK harness performance degradation with

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Attachment: profiling.png > Python SDK harness performance degradation with UnboundedThreadPoolExecutor

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Attachment: profiling_one_thread.png > Python SDK harness performance degradation with

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Issue Type: Test (was: Bug) > Python SDK harness performance degradation with

[jira] [Assigned] (BEAM-8884) Python MongoDBIO TypeError when splitting

2019-12-03 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-8884: - Assignee: Yichi Zhang > Python MongoDBIO TypeError when splitting >

[jira] [Commented] (BEAM-8884) Python MongoDBIO TypeError when splitting

2019-12-03 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987388#comment-16987388 ] Yichi Zhang commented on BEAM-8884: --- I'll take a look. > Python MongoDBIO TypeError when splitting >

[jira] [Commented] (BEAM-8884) Python MongoDBIO TypeError when splitting

2019-12-03 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987414#comment-16987414 ] Yichi Zhang commented on BEAM-8884: --- [~corvin] yeah it seems so, thanks. > Python MongoDBIO TypeError

[jira] [Created] (BEAM-8886) Add a python mongodbio integration test that triggers load split

2019-12-03 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8886: - Summary: Add a python mongodbio integration test that triggers load split Key: BEAM-8886 URL: https://issues.apache.org/jira/browse/BEAM-8886 Project: Beam Issue

[jira] [Commented] (BEAM-8884) Python MongoDBIO TypeError when splitting

2019-12-03 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987433#comment-16987433 ] Yichi Zhang commented on BEAM-8884: --- Created another Jira 

[jira] [Comment Edited] (BEAM-8884) Python MongoDBIO TypeError when splitting

2019-12-03 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987433#comment-16987433 ] Yichi Zhang edited comment on BEAM-8884 at 12/4/19 2:23 AM: Created another

[jira] [Commented] (BEAM-8884) Python MongoDBIO TypeError when splitting

2019-12-03 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987415#comment-16987415 ] Yichi Zhang commented on BEAM-8884: --- I was under the impression that the command only returns a list of

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-16 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Priority: Blocker (was: Major) > Python SDK harness performance degradation with

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-16 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Issue Type: Bug (was: Test) > Python SDK harness performance degradation with

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994104#comment-16994104 ] Yichi Zhang commented on BEAM-8944: --- Seems like the original change was to solve deadlock and stuckness

[jira] [Created] (BEAM-8824) Add support for allowed lateness in python sdk

2019-11-25 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8824: - Summary: Add support for allowed lateness in python sdk Key: BEAM-8824 URL: https://issues.apache.org/jira/browse/BEAM-8824 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-8311) Fix python mongodbio display data type

2019-09-24 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937185#comment-16937185 ] Yichi Zhang commented on BEAM-8311: --- CC: [~markflyhigh] > Fix python mongodbio display data type >

[jira] [Updated] (BEAM-8311) Fix python mongodbio display data type

2019-09-24 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8311: -- Priority: Blocker (was: Major) > Fix python mongodbio display data type >

[jira] [Updated] (BEAM-8311) Fix python mongodbio display data type

2019-09-24 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8311: -- Description: When I try to write document to mongodb through  | "Write User Doc to Mongo" >>

[jira] [Commented] (BEAM-8311) Fix python mongodbio display data type

2019-09-24 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937272#comment-16937272 ] Yichi Zhang commented on BEAM-8311: --- [https://github.com/apache/beam/pull/9656] needs to be

[jira] [Created] (BEAM-8311) Fix python mongodbio display data type

2019-09-24 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8311: - Summary: Fix python mongodbio display data type Key: BEAM-8311 URL: https://issues.apache.org/jira/browse/BEAM-8311 Project: Beam Issue Type: Bug

[jira] [Closed] (BEAM-8311) Fix python mongodbio display data type

2019-09-25 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-8311. - Resolution: Fixed > Fix python mongodbio display data type > -- > >

[jira] [Created] (BEAM-8998) Avoid excessive bundle progress polling in JRH

2019-12-19 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8998: - Summary: Avoid excessive bundle progress polling in JRH Key: BEAM-8998 URL: https://issues.apache.org/jira/browse/BEAM-8998 Project: Beam Issue Type: Improvement

[jira] [Comment Edited] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-19 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000341#comment-17000341 ] Yichi Zhang edited comment on BEAM-8944 at 12/19/19 8:14 PM: - This bug doesn't

  1   2   >