[jira] [Commented] (BEAM-10027) Support for Kotlin-based Beam Katas

2020-06-03 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125203#comment-17125203 ] Pablo Estrada commented on BEAM-10027: -- It's technically not a part of the release, so it's not that

[jira] [Updated] (BEAM-10027) Support for Kotlin-based Beam Katas

2020-06-03 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-10027: - Fix Version/s: 2.23.0 > Support for Kotlin-based Beam Katas >

[jira] [Commented] (BEAM-10068) Modify behavior of Dynamic Destinations

2020-06-01 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121408#comment-17121408 ] Pablo Estrada commented on BEAM-10068: -- Made this into a new feature -type issue. [~mborkar] can you

[jira] [Assigned] (BEAM-10068) Modify behavior of Dynamic Destinations

2020-06-01 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada reassigned BEAM-10068: Assignee: Reuven Lax (was: Pablo Estrada) > Modify behavior of Dynamic Destinations >

[jira] [Updated] (BEAM-10068) Modify behavior of Dynamic Destinations

2020-06-01 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-10068: - Issue Type: New Feature (was: Improvement) > Modify behavior of Dynamic Destinations >

[jira] [Assigned] (BEAM-10068) Modify behavior of Dynamic Destinations

2020-06-01 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada reassigned BEAM-10068: Assignee: Pablo Estrada > Modify behavior of Dynamic Destinations >

[jira] [Commented] (BEAM-1438) The default behavior for the Write transform doesn't work well with the Dataflow streaming runner

2020-05-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118985#comment-17118985 ] Pablo Estrada commented on BEAM-1438: - Ah it looks like it's just a matter of removing the check. 

[jira] [Updated] (BEAM-1438) The default behavior for the Write transform doesn't work well with the Dataflow streaming runner

2020-05-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-1438: Status: Open (was: Triage Needed) > The default behavior for the Write transform doesn't work well

[jira] [Commented] (BEAM-1438) The default behavior for the Write transform doesn't work well with the Dataflow streaming runner

2020-05-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118980#comment-17118980 ] Pablo Estrada commented on BEAM-1438: - [~reuvenlax] are you able to take a look at this? > The

[jira] [Commented] (BEAM-1438) The default behavior for the Write transform doesn't work well with the Dataflow streaming runner

2020-05-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118979#comment-17118979 ] Pablo Estrada commented on BEAM-1438: - Reopening this issue, as this will not work on Dataflow, as

[jira] [Reopened] (BEAM-1438) The default behavior for the Write transform doesn't work well with the Dataflow streaming runner

2020-05-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada reopened BEAM-1438: - > The default behavior for the Write transform doesn't work well with the > Dataflow streaming

[jira] [Assigned] (BEAM-10098) Javadoc export deactivated for RabbitMqIO and KuduIO

2020-05-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada reassigned BEAM-10098: Assignee: Pablo Estrada (was: Alexey Romanenko) > Javadoc export deactivated for

[jira] [Updated] (BEAM-10098) Javadoc export deactivated for RabbitMqIO and KuduIO

2020-05-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-10098: - Status: Open (was: Triage Needed) > Javadoc export deactivated for RabbitMqIO and KuduIO >

[jira] [Updated] (BEAM-10099) Add FhirIO and HL7v2IO to I/O matrix

2020-05-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-10099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-10099: - Status: Open (was: Triage Needed) > Add FhirIO and HL7v2IO to I/O matrix >

[jira] [Commented] (BEAM-3759) Add support for PaneInfo descriptor in Python SDK

2020-05-26 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116969#comment-17116969 ] Pablo Estrada commented on BEAM-3759: - Unfortunately this is not fixed. PR 4763 only adds encoding

[jira] [Reopened] (BEAM-3759) Add support for PaneInfo descriptor in Python SDK

2020-05-26 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada reopened BEAM-3759: - Assignee: (was: Tanay Tummalapalli) > Add support for PaneInfo descriptor in Python SDK >

[jira] [Updated] (BEAM-3767) A Complex Event Processing (CEP) library/extension for Apache Beam

2020-05-21 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-3767: Labels: gsoc gsoc2021 (was: ) > A Complex Event Processing (CEP) library/extension for Apache Beam

[jira] [Resolved] (BEAM-9148) test flakiness: BigQueryQueryToTableIT.test_big_query_standard_sql

2020-05-18 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-9148. - Fix Version/s: Not applicable Resolution: Cannot Reproduce Closing as obsolete. > test

[jira] [Resolved] (BEAM-8197) test_big_query_write_without_schema might be flaky?

2020-05-18 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-8197. - Fix Version/s: Not applicable Resolution: Cannot Reproduce Closing as obsolete. >

[jira] [Commented] (BEAM-7514) Support streaming on the Python fn_api_runner

2020-05-15 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108614#comment-17108614 ] Pablo Estrada commented on BEAM-7514: - Hi Clint! I've been working - albeit slowly- on this for a

[jira] [Resolved] (BEAM-9983) bigquery_read_it_test.ReadNewTypesTests.test_iobase_source failing

2020-05-14 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-9983. - Fix Version/s: Not applicable Resolution: Fixed this pr was reverted and rolled-forward

[jira] [Created] (BEAM-9967) Add support for BigQuery job labels on ReadFrom/WriteTo(BigQuery) transforms

2020-05-12 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9967: --- Summary: Add support for BigQuery job labels on ReadFrom/WriteTo(BigQuery) transforms Key: BEAM-9967 URL: https://issues.apache.org/jira/browse/BEAM-9967 Project: Beam

[jira] [Commented] (BEAM-6514) Dataflow Batch Job Failure is leaving Datasets/Tables behind in BigQuery

2020-05-11 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17104994#comment-17104994 ] Pablo Estrada commented on BEAM-6514: - This seems to havbe been noticed by others here: 

[jira] [Resolved] (BEAM-9957) BigQueryIO does not clean up temporary tables / dataset when a pipeline fails

2020-05-11 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-9957. - Fix Version/s: Not applicable Resolution: Duplicate BEAM-6514 > BigQueryIO does not clean

[jira] [Updated] (BEAM-9957) BigQueryIO does not clean up temporary tables / dataset when a pipeline fails

2020-05-11 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9957: Status: Open (was: Triage Needed) > BigQueryIO does not clean up temporary tables / dataset when a

[jira] [Created] (BEAM-9957) BigQueryIO does not clean up temporary tables / dataset when a pipeline fails

2020-05-11 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9957: --- Summary: BigQueryIO does not clean up temporary tables / dataset when a pipeline fails Key: BEAM-9957 URL: https://issues.apache.org/jira/browse/BEAM-9957 Project:

[jira] [Commented] (BEAM-8219) crossLanguagePortableWordCount seems to be flaky for beam_PostCommit_Python2

2020-05-11 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17104774#comment-17104774 ] Pablo Estrada commented on BEAM-8219: - Thanks Kyle! I'll close this then - it is either solved or

[jira] [Resolved] (BEAM-8219) crossLanguagePortableWordCount seems to be flaky for beam_PostCommit_Python2

2020-05-11 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-8219. - Fix Version/s: Not applicable Resolution: Fixed > crossLanguagePortableWordCount seems to

[jira] [Commented] (BEAM-8219) crossLanguagePortableWordCount seems to be flaky for beam_PostCommit_Python2

2020-05-11 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17104753#comment-17104753 ] Pablo Estrada commented on BEAM-8219: - I see this is a bit of an old issue, but I see it making Py2

[jira] [Created] (BEAM-9886) ReadFromBigQuery should auto-infer which project to bill for BQ exports

2020-05-04 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9886: --- Summary: ReadFromBigQuery should auto-infer which project to bill for BQ exports Key: BEAM-9886 URL: https://issues.apache.org/jira/browse/BEAM-9886 Project: Beam

[jira] [Resolved] (BEAM-7885) DoFn.setup() don't run for streaming jobs on DirectRunner.

2020-04-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-7885. - Fix Version/s: 2.22.0 Resolution: Fixed > DoFn.setup() don't run for streaming jobs on

[jira] [Assigned] (BEAM-7885) DoFn.setup() don't run for streaming jobs on DirectRunner.

2020-04-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada reassigned BEAM-7885: --- Assignee: Pablo Estrada > DoFn.setup() don't run for streaming jobs on DirectRunner. >

[jira] [Commented] (BEAM-7885) DoFn.setup() don't run for streaming jobs on DirectRunner.

2020-04-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095009#comment-17095009 ] Pablo Estrada commented on BEAM-7885: - This is fixed in PR 11547 > DoFn.setup() don't run for

[jira] [Commented] (BEAM-9745) [beam_PostCommit_Java_PortabilityApi] Various GCP IO tests failing, unable to deserialize Custom DoFns and Custom Coders.

2020-04-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094957#comment-17094957 ] Pablo Estrada commented on BEAM-9745: - I am trying to figure out whether this is a regression or not.

[jira] [Resolved] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-9832. - Fix Version/s: 2.22.0 Resolution: Fixed > KeyError: 'No such coder: ' in fn_runner_test >

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-28 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094700#comment-17094700 ] Pablo Estrada commented on BEAM-9832: - Okay I'll mark this as resolved then. Thanks! > KeyError: 'No

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094101#comment-17094101 ] Pablo Estrada commented on BEAM-9832: - [https://builds.apache.org/job/beam_PreCommit_Python_Cron/] 

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094091#comment-17094091 ] Pablo Estrada commented on BEAM-9832: - Nevermind. that wasn't it : ) > KeyError: 'No such coder: ' in

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094084#comment-17094084 ] Pablo Estrada commented on BEAM-9832: - I am not 100% sure, but this may be a fix: 

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094078#comment-17094078 ] Pablo Estrada commented on BEAM-9832: - hmm PR 11514 seems to only deal with monitoring infos... so I'm

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094074#comment-17094074 ] Pablo Estrada commented on BEAM-9832: - I'm suspecting this one:

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094075#comment-17094075 ] Pablo Estrada commented on BEAM-9832: - (Same test failure is visible from revert 

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094073#comment-17094073 ] Pablo Estrada commented on BEAM-9832: - Breakage seems to have started about 3 days ago: 

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094043#comment-17094043 ] Pablo Estrada commented on BEAM-9832: - Reverting PR: [https://github.com/apache/beam/pull/11544] >

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094037#comment-17094037 ] Pablo Estrada commented on BEAM-9832: - Okay, I guess it makes sense to roll back. Working on that.. >

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094030#comment-17094030 ] Pablo Estrada commented on BEAM-9832: - Hm I haven't been able to figure out how an erroneous output is

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093973#comment-17093973 ] Pablo Estrada commented on BEAM-9832: - Working on this. Interestingly enough, it's very hard to

[jira] [Commented] (BEAM-9832) KeyError: 'No such coder: ' in fn_runner_test

2020-04-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093947#comment-17093947 ] Pablo Estrada commented on BEAM-9832: - Hm sorry about the trouble. Starting to take a look... >

[jira] [Updated] (BEAM-9812) WriteToBigQuery issue causes pipelines with multiple load jobs to work erroneously

2020-04-23 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9812: Fix Version/s: 2.21.0 > WriteToBigQuery issue causes pipelines with multiple load jobs to work >

[jira] [Updated] (BEAM-9812) WriteToBigQuery issue causes pipelines with multiple load jobs to work erroneously

2020-04-23 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9812: Status: Open (was: Triage Needed) > WriteToBigQuery issue causes pipelines with multiple load jobs

[jira] [Created] (BEAM-9812) WriteToBigQuery issue causes pipelines with multiple load jobs to work erroneously

2020-04-23 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9812: --- Summary: WriteToBigQuery issue causes pipelines with multiple load jobs to work erroneously Key: BEAM-9812 URL: https://issues.apache.org/jira/browse/BEAM-9812

[jira] [Commented] (BEAM-9745) [beam_PostCommit_Java_PortabilityApi] Various GCP IO tests failing, unable to deserialize Custom DoFns and Custom Coders.

2020-04-22 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17090157#comment-17090157 ] Pablo Estrada commented on BEAM-9745: - Nevermind, on a723148858b421f7df08c74f11432b4aeb2f561c the test

[jira] [Commented] (BEAM-9745) [beam_PostCommit_Java_PortabilityApi] Various GCP IO tests failing, unable to deserialize Custom DoFns and Custom Coders.

2020-04-22 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17090105#comment-17090105 ] Pablo Estrada commented on BEAM-9745: - FWIW, this issue only seems to be affecting the Java SDK

[jira] [Created] (BEAM-9787) Send clear error to users trying to use BigQuerySource on FnApi pipelines on Python SDK

2020-04-20 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9787: --- Summary: Send clear error to users trying to use BigQuerySource on FnApi pipelines on Python SDK Key: BEAM-9787 URL: https://issues.apache.org/jira/browse/BEAM-9787

[jira] [Comment Edited] (BEAM-9745) [beam_PostCommit_Java_PortabilityApi] Various GCP IO tests failing, unable to deserialize Custom DoFns and Custom Coders.

2020-04-17 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086076#comment-17086076 ] Pablo Estrada edited comment on BEAM-9745 at 4/17/20, 9:24 PM: --- This suite

[jira] [Updated] (BEAM-9769) Ensure JSON imports are the default behavior for BigQuerySink and WriteToBigQuery in Python

2020-04-15 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9769: Fix Version/s: 2.21.0 > Ensure JSON imports are the default behavior for BigQuerySink and >

[jira] [Updated] (BEAM-9769) Ensure JSON imports are the default behavior for BigQuerySink and WriteToBigQuery in Python

2020-04-15 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9769: Status: Open (was: Triage Needed) > Ensure JSON imports are the default behavior for BigQuerySink

[jira] [Created] (BEAM-9769) Ensure JSON imports are the default behavior for BigQuerySink and WriteToBigQuery in Python

2020-04-15 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9769: --- Summary: Ensure JSON imports are the default behavior for BigQuerySink and WriteToBigQuery in Python Key: BEAM-9769 URL: https://issues.apache.org/jira/browse/BEAM-9769

[jira] [Updated] (BEAM-9763) Make _ReadFromBigQuery public

2020-04-15 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9763: Description: This means removing the underscore from it, but keeping it tagged as experimental. >

[jira] [Created] (BEAM-9763) Make _ReadFromBigQuery public

2020-04-15 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9763: --- Summary: Make _ReadFromBigQuery public Key: BEAM-9763 URL: https://issues.apache.org/jira/browse/BEAM-9763 Project: Beam Issue Type: Sub-task

[jira] [Updated] (BEAM-9762) Production-ready BigQuery IOs for Python SDK

2020-04-15 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9762: Status: Open (was: Triage Needed) > Production-ready BigQuery IOs for Python SDK >

[jira] [Updated] (BEAM-9763) Make _ReadFromBigQuery public

2020-04-15 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9763: Status: Open (was: Triage Needed) > Make _ReadFromBigQuery public > -

[jira] [Created] (BEAM-9762) Production-ready BigQuery IOs for Python SDK

2020-04-15 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9762: --- Summary: Production-ready BigQuery IOs for Python SDK Key: BEAM-9762 URL: https://issues.apache.org/jira/browse/BEAM-9762 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-9741) Split timers to multiple workers in ParallelBundleManager

2020-04-10 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9741: --- Summary: Split timers to multiple workers in ParallelBundleManager Key: BEAM-9741 URL: https://issues.apache.org/jira/browse/BEAM-9741 Project: Beam Issue

[jira] [Commented] (BEAM-9494) Remove workaround for BQ transform for Dataflow

2020-04-08 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17078776#comment-17078776 ] Pablo Estrada commented on BEAM-9494: - Not resolved, but we can remove the fix version. > Remove

[jira] [Updated] (BEAM-9494) Remove workaround for BQ transform for Dataflow

2020-04-08 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9494: Fix Version/s: (was: 2.21.0) > Remove workaround for BQ transform for Dataflow >

[jira] [Resolved] (BEAM-9691) Ensure Dataflow BQ Native sink are not used on FnApi

2020-04-07 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-9691. - Fix Version/s: 2.21.0 Resolution: Fixed > Ensure Dataflow BQ Native sink are not used on

[jira] [Updated] (BEAM-9715) annotations_test fails in some environmens

2020-04-06 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9715: Priority: Minor (was: Major) > annotations_test fails in some environmens >

[jira] [Updated] (BEAM-9715) annotations_test fails in some environmens

2020-04-06 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9715: Status: Open (was: Triage Needed) > annotations_test fails in some environmens >

[jira] [Created] (BEAM-9715) annotations_test fails in some environmens

2020-04-06 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9715: --- Summary: annotations_test fails in some environmens Key: BEAM-9715 URL: https://issues.apache.org/jira/browse/BEAM-9715 Project: Beam Issue Type: Bug

[jira] [Updated] (BEAM-9691) Ensure Dataflow BQ Native sink are not used on FnApi

2020-04-03 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9691: Status: Open (was: Triage Needed) > Ensure Dataflow BQ Native sink are not used on FnApi >

[jira] [Commented] (BEAM-9691) Ensure Dataflow BQ Native sink are not used on FnApi

2020-04-03 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074895#comment-17074895 ] Pablo Estrada commented on BEAM-9691: - Ah, in fact this is for native sink, not source. I have a PR

[jira] [Updated] (BEAM-9691) Ensure Dataflow BQ Native sink are not used on FnApi

2020-04-03 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9691: Summary: Ensure Dataflow BQ Native sink are not used on FnApi (was: Ensure Dataflow BQ Native

[jira] [Commented] (BEAM-9691) Ensure Dataflow BQ Native sources are not used on FnApi

2020-04-03 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074892#comment-17074892 ] Pablo Estrada commented on BEAM-9691: - I think the title of the issue is too wide. Sorry about that.

[jira] [Updated] (BEAM-9691) Ensure Dataflow BQ Native sources are not used on FnApi

2020-04-03 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9691: Summary: Ensure Dataflow BQ Native sources are not used on FnApi (was: Ensure Dataflow Native

[jira] [Updated] (BEAM-9691) Ensure Dataflow Native sources are not used on FnApi

2020-04-03 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9691: Summary: Ensure Dataflow Native sources are not used on FnApi (was: Ensure Dataflow Native sources

[jira] [Created] (BEAM-9691) Ensure Dataflow Native sources are not used on FnApi ever

2020-04-03 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9691: --- Summary: Ensure Dataflow Native sources are not used on FnApi ever Key: BEAM-9691 URL: https://issues.apache.org/jira/browse/BEAM-9691 Project: Beam Issue

[jira] [Created] (BEAM-9640) Track PCollection watermark across bundle executions

2020-03-30 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9640: --- Summary: Track PCollection watermark across bundle executions Key: BEAM-9640 URL: https://issues.apache.org/jira/browse/BEAM-9640 Project: Beam Issue Type:

[jira] [Updated] (BEAM-9639) Abstract bundle execution logic from stage execution logic

2020-03-30 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9639: Status: Open (was: Triage Needed) > Abstract bundle execution logic from stage execution logic >

[jira] [Created] (BEAM-9639) Abstract bundle execution logic from stage execution logic

2020-03-30 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9639: --- Summary: Abstract bundle execution logic from stage execution logic Key: BEAM-9639 URL: https://issues.apache.org/jira/browse/BEAM-9639 Project: Beam Issue

[jira] [Resolved] (BEAM-9608) Add context managers for FnApiRunner to manage execution of each bundle

2020-03-30 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-9608. - Fix Version/s: 2.21.0 Resolution: Fixed > Add context managers for FnApiRunner to manage

[jira] [Updated] (BEAM-9625) StateServicer should be owned by FnApiRunnerContextManager

2020-03-27 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9625: Status: Open (was: Triage Needed) > StateServicer should be owned by FnApiRunnerContextManager >

[jira] [Created] (BEAM-9625) StateServicer should be owned by FnApiRunnerContextManager

2020-03-27 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9625: --- Summary: StateServicer should be owned by FnApiRunnerContextManager Key: BEAM-9625 URL: https://issues.apache.org/jira/browse/BEAM-9625 Project: Beam Issue

[jira] [Updated] (BEAM-9608) Add context managers for FnApiRunner to manage execution of each bundle

2020-03-25 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9608: Status: Open (was: Triage Needed) > Add context managers for FnApiRunner to manage execution of

[jira] [Resolved] (BEAM-9537) Refactor FnApiRunner into its own package

2020-03-25 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-9537. - Fix Version/s: 2.21.0 Resolution: Fixed > Refactor FnApiRunner into its own package >

[jira] [Created] (BEAM-9608) Add context managers for FnApiRunner to manage execution of each bundle

2020-03-25 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9608: --- Summary: Add context managers for FnApiRunner to manage execution of each bundle Key: BEAM-9608 URL: https://issues.apache.org/jira/browse/BEAM-9608 Project: Beam

[jira] [Created] (BEAM-9601) Interactive test_streaming_wordcount failing

2020-03-24 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9601: --- Summary: Interactive test_streaming_wordcount failing Key: BEAM-9601 URL: https://issues.apache.org/jira/browse/BEAM-9601 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-9598) _CustomBigQuerySource checks valueprovider when it's not needed

2020-03-24 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9598: --- Summary: _CustomBigQuerySource checks valueprovider when it's not needed Key: BEAM-9598 URL: https://issues.apache.org/jira/browse/BEAM-9598 Project: Beam

[jira] [Updated] (BEAM-9572) Allow RuntimeValueProviders for WriteToBigQuery transform (dataflow templates)

2020-03-24 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9572: Priority: Minor (was: Critical) > Allow RuntimeValueProviders for WriteToBigQuery transform

[jira] [Commented] (BEAM-9572) Allow RuntimeValueProviders for WriteToBigQuery transform (dataflow templates)

2020-03-24 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066016#comment-17066016 ] Pablo Estrada commented on BEAM-9572: - Did you pass the pipeline option

[jira] [Resolved] (BEAM-9572) Allow RuntimeValueProviders for WriteToBigQuery transform (dataflow templates)

2020-03-24 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada resolved BEAM-9572. - Fix Version/s: Not applicable Resolution: Fixed > Allow RuntimeValueProviders for

[jira] [Updated] (BEAM-9572) Allow RuntimeValueProviders for WriteToBigQuery transform (dataflow templates)

2020-03-24 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9572: Component/s: (was: beam-model) (was: beam-community)

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

2020-03-18 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17061974#comment-17061974 ] Pablo Estrada commented on BEAM-2572: - Thanks for catching that Badrul! Would you file a JIRA to

[jira] [Created] (BEAM-9537) Refactor FnApiRunner into its own package

2020-03-17 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9537: --- Summary: Refactor FnApiRunner into its own package Key: BEAM-9537 URL: https://issues.apache.org/jira/browse/BEAM-9537 Project: Beam Issue Type: Sub-task

[jira] [Created] (BEAM-9533) Replace *-gcp/*-aws tox suites with *-cloud suites to run unit tests for both

2020-03-17 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9533: --- Summary: Replace *-gcp/*-aws tox suites with *-cloud suites to run unit tests for both Key: BEAM-9533 URL: https://issues.apache.org/jira/browse/BEAM-9533 Project:

[jira] [Comment Edited] (BEAM-9484) apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests is flaky in DirectRunner Postcommits

2020-03-17 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17061257#comment-17061257 ] Pablo Estrada edited comment on BEAM-9484 at 3/17/20, 11:34 PM:

[jira] [Commented] (BEAM-9484) apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests is flaky in DirectRunner Postcommits

2020-03-17 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17061257#comment-17061257 ] Pablo Estrada commented on BEAM-9484: - [~chamikara] lmk if you need a hand looking into this, or if

[jira] [Commented] (BEAM-9532) test_last_updated is failing in s3io_test

2020-03-17 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17061253#comment-17061253 ] Pablo Estrada commented on BEAM-9532: - cc: [~mattmorgis] fyi > test_last_updated is failing in

[jira] [Created] (BEAM-9532) test_last_updated is failing in s3io_test

2020-03-17 Thread Pablo Estrada (Jira)
Pablo Estrada created BEAM-9532: --- Summary: test_last_updated is failing in s3io_test Key: BEAM-9532 URL: https://issues.apache.org/jira/browse/BEAM-9532 Project: Beam Issue Type: Bug

[jira] [Updated] (BEAM-9494) Remove workaround for BQ transform for Dataflow

2020-03-17 Thread Pablo Estrada (Jira)
[ https://issues.apache.org/jira/browse/BEAM-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo Estrada updated BEAM-9494: Fix Version/s: (was: 2.20.0) 2.21.0 > Remove workaround for BQ transform for

  1   2   3   4   5   >