[jira] [Created] (BEAM-10189) Add ValueState to python sdk

2020-06-03 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-10189: -- Summary: Add ValueState to python sdk Key: BEAM-10189 URL: https://issues.apache.org/jira/browse/BEAM-10189 Project: Beam Issue Type: Improvement

[jira] [Resolved] (BEAM-9603) Support Dynamic Timer in Java SDK over FnApi

2020-06-02 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-9603. --- Fix Version/s: 2.22.0 Resolution: Fixed > Support Dynamic Timer in Java SDK over FnApi >

[jira] [Resolved] (BEAM-9263) Bump python sdk fnapi version to enable status reporting

2020-06-02 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-9263. --- Fix Version/s: 2.20.0 Resolution: Fixed > Bump python sdk fnapi version to enable status

[jira] [Updated] (BEAM-10153) Java SDK Harness throws NPE for VoidCoder as key coder

2020-05-29 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-10153: --- Description: When using VoidCoder as the key coder, NullPointerException will be thrown for

[jira] [Created] (BEAM-10153) Java SDK Harness throws NPE for VoidCoder for key coder

2020-05-29 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-10153: -- Summary: Java SDK Harness throws NPE for VoidCoder for key coder Key: BEAM-10153 URL: https://issues.apache.org/jira/browse/BEAM-10153 Project: Beam Issue Type:

[jira] [Updated] (BEAM-10153) Java SDK Harness throws NPE for VoidCoder as key coder

2020-05-29 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-10153: --- Summary: Java SDK Harness throws NPE for VoidCoder as key coder (was: Java SDK Harness throws NPE

[jira] [Assigned] (BEAM-10112) Add python sdk state and timer examples to website

2020-05-27 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-10112: -- Assignee: Yichi Zhang > Add python sdk state and timer examples to website >

[jira] [Created] (BEAM-10112) Add python sdk state and timer examples to website

2020-05-27 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-10112: -- Summary: Add python sdk state and timer examples to website Key: BEAM-10112 URL: https://issues.apache.org/jira/browse/BEAM-10112 Project: Beam Issue Type:

[jira] [Assigned] (BEAM-9602) Support Dynamic Timer in Python SDK over FnApi

2020-05-26 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-9602: - Assignee: Yichi Zhang > Support Dynamic Timer in Python SDK over FnApi >

[jira] [Commented] (BEAM-10002) Mongo cursor timeout leads to CursorNotFound error

2020-05-15 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108429#comment-17108429 ] Yichi Zhang commented on BEAM-10002: Thanks for the issue report [~corvin], do you want also

[jira] [Created] (BEAM-9940) Dataflow runner not setting timer family specs for TimerDeclaration annotation

2020-05-08 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9940: - Summary: Dataflow runner not setting timer family specs for TimerDeclaration annotation Key: BEAM-9940 URL: https://issues.apache.org/jira/browse/BEAM-9940 Project: Beam

[jira] [Closed] (BEAM-9904) python nexmark query pipelines

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-9904. - Fix Version/s: Not applicable Resolution: Abandoned > python nexmark query pipelines >

[jira] [Closed] (BEAM-9905) python nexmark benchmark suite metrics

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-9905. - Fix Version/s: Not applicable Resolution: Abandoned > python nexmark benchmark suite metrics >

[jira] [Closed] (BEAM-9902) python nexmark benchmark suite event generator

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-9902. - Fix Version/s: Not applicable Resolution: Abandoned > python nexmark benchmark suite event

[jira] [Closed] (BEAM-9903) python nexmark benchmark suite launcher

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-9903. - Fix Version/s: Not applicable Resolution: Abandoned > python nexmark benchmark suite launcher >

[jira] [Closed] (BEAM-9901) Beam python nexmark benchmark suite

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-9901. - Fix Version/s: Not applicable Resolution: Resolved > Beam python nexmark benchmark suite >

[jira] [Closed] (BEAM-9906) Beam python nexmark benchmark starter task

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-9906. - Fix Version/s: Not applicable Resolution: Abandoned > Beam python nexmark benchmark starter task >

[jira] [Commented] (BEAM-9901) Beam python nexmark benchmark suite

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17101856#comment-17101856 ] Yichi Zhang commented on BEAM-9901: --- Ok, I guess we can use BEAM-8258 for the summer intern as well.

[jira] [Commented] (BEAM-9905) python nexmark benchmark suite metrics

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17101848#comment-17101848 ] Yichi Zhang commented on BEAM-9905: --- [~iemejia]It is more about being able to run the queries with test

[jira] [Updated] (BEAM-9905) python nexmark benchmark suite metrics

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9905: -- Description: Ensure we can collect output metrics from the query pipelines with the jenkins test infra

[jira] [Commented] (BEAM-9901) Beam python nexmark benchmark suite

2020-05-07 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17101845#comment-17101845 ] Yichi Zhang commented on BEAM-9901: --- [~iemejia] We are creating these tickets for google summer intern

[jira] [Updated] (BEAM-9906) Beam python nextmark benchmark starter task

2020-05-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9906: -- Description: For this starter task, the assignee is expected to * get familiar with beam model by

[jira] [Created] (BEAM-9906) Beam python nextmark benchmark starter task

2020-05-06 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9906: - Summary: Beam python nextmark benchmark starter task Key: BEAM-9906 URL: https://issues.apache.org/jira/browse/BEAM-9906 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-9905) python nexmark benchmark suite metrics

2020-05-06 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9905: - Summary: python nexmark benchmark suite metrics Key: BEAM-9905 URL: https://issues.apache.org/jira/browse/BEAM-9905 Project: Beam Issue Type: Sub-task

[jira] [Updated] (BEAM-9904) python nexmark query pipelines

2020-05-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9904: -- Description: Create python beam pipelines for the data analytic queries in NexMark suite:  

[jira] [Updated] (BEAM-9904) python nexmark query pipelines

2020-05-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9904: -- Description: Create python beam pipelines for the data analytic queries in NexMark suite:  

[jira] [Updated] (BEAM-9904) python nexmark query pipelines

2020-05-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9904: -- Description:   ||query||java code|| |*Query 0 (not part of original NexMark):

[jira] [Updated] (BEAM-9904) python nexmark query pipelines

2020-05-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9904: -- Description:   ||query||java code|| |*Query 0 (not part of original NexMark):

[jira] [Updated] (BEAM-9904) python nexmark query pipelines

2020-05-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9904: -- Description:   ||query||java code|| |*Query 0 (not part of original NexMark):

[jira] [Updated] (BEAM-9904) python nexmark query pipelines

2020-05-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9904: -- Description:   ||query||java code|| |*Query 0 (not part of original NexMark):

[jira] [Created] (BEAM-9904) python nexmark query pipelines

2020-05-06 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9904: - Summary: python nexmark query pipelines Key: BEAM-9904 URL: https://issues.apache.org/jira/browse/BEAM-9904 Project: Beam Issue Type: Sub-task

[jira] [Updated] (BEAM-9901) Beam python nexmark benchmark suite

2020-05-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9901: -- Component/s: (was: sdk-py-core) > Beam python nexmark benchmark suite >

[jira] [Updated] (BEAM-9902) python nexmark benchmark suite event generator

2020-05-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9902: -- Component/s: (was: sdk-py-core) > python nexmark benchmark suite event generator >

[jira] [Created] (BEAM-9903) python nexmark benchmark suite launcher

2020-05-06 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9903: - Summary: python nexmark benchmark suite launcher Key: BEAM-9903 URL: https://issues.apache.org/jira/browse/BEAM-9903 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-9902) python nexmark benchmark suite event generator

2020-05-06 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9902: - Summary: python nexmark benchmark suite event generator Key: BEAM-9902 URL: https://issues.apache.org/jira/browse/BEAM-9902 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-9901) Beam python nexmark benchmark suite

2020-05-06 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9901: - Summary: Beam python nexmark benchmark suite Key: BEAM-9901 URL: https://issues.apache.org/jira/browse/BEAM-9901 Project: Beam Issue Type: Task

[jira] [Created] (BEAM-9727) Auto populate required feature experiment flags for enable dataflow runner v2

2020-04-08 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9727: - Summary: Auto populate required feature experiment flags for enable dataflow runner v2 Key: BEAM-9727 URL: https://issues.apache.org/jira/browse/BEAM-9727 Project: Beam

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-03-31 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071998#comment-17071998 ] Yichi Zhang commented on BEAM-8944: --- [~mxm] the checkpoint duration is lower means that without

[jira] [Commented] (BEAM-9649) beam_python_mongoio_load_test started failing due to mismatched results

2020-03-31 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071971#comment-17071971 ] Yichi Zhang commented on BEAM-9649: --- cc: +[~chamikara] fyi Not particularly sure when it started

[jira] [Updated] (BEAM-9649) beam_python_mongoio_load_test started failing due to mismatched results

2020-03-31 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9649: -- Priority: Critical (was: Minor) > beam_python_mongoio_load_test started failing due to mismatched

[jira] [Updated] (BEAM-9649) beam_python_mongoio_load_test started failing due to mismatched results

2020-03-31 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9649: -- Attachment: mHP2wb3rdTG.png > beam_python_mongoio_load_test started failing due to mismatched results >

[jira] [Updated] (BEAM-9649) beam_python_mongoio_load_test started failing due to mismatched results

2020-03-31 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9649: -- Description: The load tests fail sometimes with a mismatched sum result for example

[jira] [Updated] (BEAM-9649) beam_python_mongoio_load_test started failing due to mismatched results

2020-03-31 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9649: -- Attachment: j5vwSDNmTBK.png > beam_python_mongoio_load_test started failing due to mismatched results >

[jira] [Updated] (BEAM-9649) beam_python_mongoio_load_test started failing due to mismatched results

2020-03-31 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9649: -- Summary: beam_python_mongoio_load_test started failing due to mismatched results (was:

[jira] [Updated] (BEAM-9649) beam_python_mongoio_load_test started failing due to one mismatched results

2020-03-31 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9649: -- Description: The load tests fail sometimes with a mismatched sum result for example

[jira] [Created] (BEAM-9649) beam_python_mongoio_load_test started failing due to one mismatched results

2020-03-31 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9649: - Summary: beam_python_mongoio_load_test started failing due to one mismatched results Key: BEAM-9649 URL: https://issues.apache.org/jira/browse/BEAM-9649 Project: Beam

[jira] [Created] (BEAM-9607) _SDFBoundedSourceWrapper should expose underlying source display_data

2020-03-25 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9607: - Summary: _SDFBoundedSourceWrapper should expose underlying source display_data Key: BEAM-9607 URL: https://issues.apache.org/jira/browse/BEAM-9607 Project: Beam

[jira] [Closed] (BEAM-9336) beam_PostCommit_Py_ValCont tests timeout

2020-02-21 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-9336. - Fix Version/s: Not applicable Resolution: Duplicate > beam_PostCommit_Py_ValCont tests timeout >

[jira] [Created] (BEAM-9336) beam_PostCommit_Py_ValCont tests timeout

2020-02-19 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9336: - Summary: beam_PostCommit_Py_ValCont tests timeout Key: BEAM-9336 URL: https://issues.apache.org/jira/browse/BEAM-9336 Project: Beam Issue Type: Bug

[jira] [Updated] (BEAM-9336) beam_PostCommit_Py_ValCont tests timeout

2020-02-19 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-9336: -- Description:   * [[https://builds.apache.org/job/beam_PostCommit_Py_ValCont/]] Initial investigation:

[jira] [Closed] (BEAM-9334) beam_PreCommit_Java_Cron failed on org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactoryTest.testProcessingTimeTimers

2020-02-18 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang closed BEAM-9334. - Fix Version/s: Not applicable Resolution: Duplicate > beam_PreCommit_Java_Cron failed on >

[jira] [Created] (BEAM-9334) beam_PreCommit_Java_Cron failed on org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactoryTest.testProcessingTimeTimers

2020-02-18 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9334: - Summary: beam_PreCommit_Java_Cron failed on org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactoryTest.testProcessingTimeTimers Key: BEAM-9334 URL:

[jira] [Created] (BEAM-9263) Bump python sdk fnapi version to enable status reporting

2020-02-06 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9263: - Summary: Bump python sdk fnapi version to enable status reporting Key: BEAM-9263 URL: https://issues.apache.org/jira/browse/BEAM-9263 Project: Beam Issue Type:

[jira] [Resolved] (BEAM-8614) Expose SDK harness status to Runner through FnApi

2020-02-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8614. --- Fix Version/s: 2.20.0 Resolution: Fixed > Expose SDK harness status to Runner through FnApi >

[jira] [Resolved] (BEAM-9122) Add uses_keyed_state step property to python dataflow runner

2020-02-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-9122. --- Fix Version/s: 2.19.0 Resolution: Fixed > Add uses_keyed_state step property to python

[jira] [Resolved] (BEAM-8626) Implement status api handler in python sdk harness

2020-02-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8626. --- Fix Version/s: 2.20.0 Resolution: Fixed > Implement status api handler in python sdk harness >

[jira] [Resolved] (BEAM-8625) Implement servlet in Dataflow runner for sdk status query endpoint

2020-02-06 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8625. --- Fix Version/s: 2.20.0 Resolution: Fixed > Implement servlet in Dataflow runner for sdk status

[jira] [Created] (BEAM-9165) Clean up StatusServer from python sdk harness when status reporting over fnapi is enabled

2020-01-21 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9165: - Summary: Clean up StatusServer from python sdk harness when status reporting over fnapi is enabled Key: BEAM-9165 URL: https://issues.apache.org/jira/browse/BEAM-9165

[jira] [Created] (BEAM-9122) Add uses_keyed_state step property to python dataflow runner

2020-01-14 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-9122: - Summary: Add uses_keyed_state step property to python dataflow runner Key: BEAM-9122 URL: https://issues.apache.org/jira/browse/BEAM-9122 Project: Beam Issue

[jira] [Assigned] (BEAM-9122) Add uses_keyed_state step property to python dataflow runner

2020-01-14 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-9122: - Assignee: Yichi Zhang > Add uses_keyed_state step property to python dataflow runner >

[jira] [Resolved] (BEAM-8624) Implement FnService for status api in Dataflow runner

2020-01-14 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8624. --- Fix Version/s: 2.19.0 Resolution: Fixed > Implement FnService for status api in Dataflow

[jira] [Resolved] (BEAM-8824) Add support for allowed lateness in python sdk

2020-01-14 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8824. --- Fix Version/s: 2.18.0 Resolution: Fixed > Add support for allowed lateness in python sdk >

[jira] [Resolved] (BEAM-8736) Support window allowed lateness in python sdk

2020-01-14 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8736. --- Fix Version/s: 2.18.0 Resolution: Fixed > Support window allowed lateness in python sdk >

[jira] [Resolved] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-26 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8944. --- Resolution: Fixed Mitigation is merged, future investigation of how to improve performance will be

[jira] [Work started] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-26 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on BEAM-8944 started by Yichi Zhang. - > Python SDK harness performance degradation with UnboundedThreadPoolExecutor >

[jira] [Work started] (BEAM-8623) Add additional message field to Provision API response for passing status endpoint

2019-12-23 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on BEAM-8623 started by Yichi Zhang. - > Add additional message field to Provision API response for passing status >

[jira] [Resolved] (BEAM-8623) Add additional message field to Provision API response for passing status endpoint

2019-12-23 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8623. --- Fix Version/s: 2.19.0 Resolution: Fixed > Add additional message field to Provision API

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-21 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001802#comment-17001802 ] Yichi Zhang commented on BEAM-8944: --- I think so > Python SDK harness performance degradation with

[jira] [Commented] (BEAM-8998) Avoid excessive bundle progress polling in Dataflow Runner

2019-12-20 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001079#comment-17001079 ] Yichi Zhang commented on BEAM-8998: --- throttling is introduced to avoid expensive scheduling problem

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-19 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000448#comment-17000448 ] Yichi Zhang commented on BEAM-8944: --- then yeah, it'll affect python streaming jobs (which is only on

[jira] [Comment Edited] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-19 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000341#comment-17000341 ] Yichi Zhang edited comment on BEAM-8944 at 12/19/19 8:14 PM: - This bug doesn't

[jira] [Updated] (BEAM-8998) Avoid excessive bundle progress polling in Dataflow Runner

2019-12-19 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8998: -- Description: Dataflow Java runner uses 0.1 secs interval for polling bundle progress from SDK Harness,

[jira] [Updated] (BEAM-8998) Avoid excessive bundle progress polling in Dataflow Runner

2019-12-19 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8998: -- Description: Dataflow Java runner uses 0.1 secs interval for polling bundle progress from SDK Harness,

[jira] [Updated] (BEAM-8998) Avoid excessive bundle progress polling in Dataflow Runner

2019-12-19 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8998: -- Summary: Avoid excessive bundle progress polling in Dataflow Runner (was: Avoid excessive bundle

[jira] [Created] (BEAM-8998) Avoid excessive bundle progress polling in JRH

2019-12-19 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8998: - Summary: Avoid excessive bundle progress polling in JRH Key: BEAM-8998 URL: https://issues.apache.org/jira/browse/BEAM-8998 Project: Beam Issue Type: Improvement

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-16 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Priority: Blocker (was: Major) > Python SDK harness performance degradation with

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-16 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Issue Type: Bug (was: Test) > Python SDK harness performance degradation with

[jira] [Resolved] (BEAM-8886) Add a python mongodbio integration test that triggers load split

2019-12-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8886. --- Fix Version/s: 2.19.0 Resolution: Fixed > Add a python mongodbio integration test that

[jira] [Resolved] (BEAM-8884) Python MongoDBIO TypeError when splitting

2019-12-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang resolved BEAM-8884. --- Resolution: Fixed > Python MongoDBIO TypeError when splitting >

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-12 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Issue Type: Test (was: Bug) > Python SDK harness performance degradation with

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994104#comment-16994104 ] Yichi Zhang commented on BEAM-8944: --- Seems like the original change was to solve deadlock and stuckness

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Attachment: profiling_twelve_threads.png > Python SDK harness performance degradation with

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Attachment: profiling_one_thread.png > Python SDK harness performance degradation with

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Attachment: profiling.png > Python SDK harness performance degradation with UnboundedThreadPoolExecutor

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Affects Version/s: (was: 2.17.0) > Python SDK harness performance degradation with

[jira] [Comment Edited] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993141#comment-16993141 ] Yichi Zhang edited comment on BEAM-8944 at 12/11/19 2:28 AM: - CC: [~angoenka]

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993141#comment-16993141 ] Yichi Zhang commented on BEAM-8944: --- CC: [~angoenka] > Python SDK harness performance degradation with

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Updated] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang updated BEAM-8944: -- Description: We are seeing a performance degradation for python streaming word count load tests.  

[jira] [Created] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
Yichi Zhang created BEAM-8944: - Summary: Python SDK harness performance degradation with UnboundedThreadPoolExecutor Key: BEAM-8944 URL: https://issues.apache.org/jira/browse/BEAM-8944 Project: Beam

[jira] [Assigned] (BEAM-8580) Request Python API to support windows ClosingBehavior

2019-12-09 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yichi Zhang reassigned BEAM-8580: - Assignee: Yichi Zhang > Request Python API to support windows ClosingBehavior >

[jira] [Comment Edited] (BEAM-8884) Python MongoDBIO TypeError when splitting

2019-12-03 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987433#comment-16987433 ] Yichi Zhang edited comment on BEAM-8884 at 12/4/19 2:23 AM: Created another

  1   2   >