[jira] [Resolved] (BEAM-3991) Update dependency 'google-api-services-storage' to latest version

[ https://issues.apache.org/jira/browse/BEAM-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-3991. -- Resolution: Fixed > Update dependency 'google-api-services-storage' to latest version >

[jira] [Commented] (BEAM-3991) Update dependency 'google-api-services-storage' to latest version

[ https://issues.apache.org/jira/browse/BEAM-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444777#comment-16444777 ] Chamikara Jayalath commented on BEAM-3991: -- [https://github.com/apache/beam/pull/5046] is in

[jira] [Created] (BEAM-4133) Properly handle HTTP 308 errors from GCS

Chamikara Jayalath created BEAM-4133: Summary: Properly handle HTTP 308 errors from GCS Key: BEAM-4133 URL: https://issues.apache.org/jira/browse/BEAM-4133 Project: Beam Issue Type:

[jira] [Commented] (BEAM-4096) BigQueryIO ValueProvider support for Method and Triggering Frequency

[ https://issues.apache.org/jira/browse/BEAM-4096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440217#comment-16440217 ] Chamikara Jayalath commented on BEAM-4096: -- Looks like you haven't been given the Jira

[jira] [Assigned] (BEAM-4096) BigQueryIO ValueProvider support for Method and Triggering Frequency

[ https://issues.apache.org/jira/browse/BEAM-4096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-4096: Assignee: (was: Chamikara Jayalath) > BigQueryIO ValueProvider support for

[jira] [Commented] (BEAM-4096) BigQueryIO ValueProvider support for Method and Triggering Frequency

[ https://issues.apache.org/jira/browse/BEAM-4096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440214#comment-16440214 ] Chamikara Jayalath commented on BEAM-4096: -- Ryan, thanks for filing the JIRA. Is this something

[jira] [Commented] (BEAM-3991) Update dependency 'google-api-services-storage' to latest version

[ https://issues.apache.org/jira/browse/BEAM-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434750#comment-16434750 ] Chamikara Jayalath commented on BEAM-3991: -- Python SDK was updated. PR for updating Java SDK is

[jira] [Updated] (BEAM-4055) Migrate Python SDK off of apitools dependency

[ https://issues.apache.org/jira/browse/BEAM-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-4055: - Summary: Migrate Python SDK off of apitools dependency (was: Migrate Python SDK off

[jira] [Commented] (BEAM-4055) Migrate Python SDK off apitools

[ https://issues.apache.org/jira/browse/BEAM-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434667#comment-16434667 ] Chamikara Jayalath commented on BEAM-4055: -- cc: [~altay] > Migrate Python SDK off apitools >

[jira] [Created] (BEAM-4055) Migrate Python SDK off apitools

Chamikara Jayalath created BEAM-4055: Summary: Migrate Python SDK off apitools Key: BEAM-4055 URL: https://issues.apache.org/jira/browse/BEAM-4055 Project: Beam Issue Type: Improvement

[jira] [Commented] (BEAM-3991) Update dependency 'google-api-services-storage' to latest version

[ https://issues.apache.org/jira/browse/BEAM-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434425#comment-16434425 ] Chamikara Jayalath commented on BEAM-3991: -- Due to global batch endpoint service discontinuation

[jira] [Updated] (BEAM-3991) Update dependency 'google-api-services-storage' to latest version

[ https://issues.apache.org/jira/browse/BEAM-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3991: - Component/s: (was: sdk-py-core) > Update dependency 'google-api-services-storage' to

[jira] [Updated] (BEAM-3991) Update dependency 'google-api-services-storage' to latest version

[ https://issues.apache.org/jira/browse/BEAM-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3991: - Component/s: sdk-py-core > Update dependency 'google-api-services-storage' to latest

[jira] [Updated] (BEAM-3991) Update dependency 'google-api-services-storage' to latest version

[ https://issues.apache.org/jira/browse/BEAM-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3991: - Component/s: sdk-py-core > Update dependency 'google-api-services-storage' to latest

[jira] [Commented] (BEAM-1191) Eliminate OldDoFn from the SDK

[ https://issues.apache.org/jira/browse/BEAM-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433130#comment-16433130 ] Chamikara Jayalath commented on BEAM-1191: -- What is the status of this ? > Eliminate OldDoFn from

[jira] [Resolved] (BEAM-2403) Provide BigQueryIO with method to indicate Priority (INTERACTIVE/BATCH)

[ https://issues.apache.org/jira/browse/BEAM-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-2403. -- Resolution: Duplicate Fix Version/s: 2.5.0 Duplicate of 

[jira] [Commented] (BEAM-2403) Provide BigQueryIO with method to indicate Priority (INTERACTIVE/BATCH)

[ https://issues.apache.org/jira/browse/BEAM-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431531#comment-16431531 ] Chamikara Jayalath commented on BEAM-2403: -- Yeah, marking this issue as a duplicate. > Provide

[jira] [Commented] (BEAM-3991) Update dependency 'google-api-services-storage' to latest version

[ https://issues.apache.org/jira/browse/BEAM-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16428763#comment-16428763 ] Chamikara Jayalath commented on BEAM-3991: -- Due to conflicting transitive dependencies we have to

[jira] [Resolved] (BEAM-3774) Update BigQuery jobs to explicitly specify the region

[ https://issues.apache.org/jira/browse/BEAM-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-3774. -- Resolution: Fixed Fix Version/s: 2.5.0 > Update BigQuery jobs to explicitly

[jira] [Assigned] (BEAM-3980) Performance tests are failing due to AttributeError: run_with_pdb

[ https://issues.apache.org/jira/browse/BEAM-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3980: Assignee: Łukasz Gajowy > Performance tests are failing due to AttributeError:

[jira] [Resolved] (BEAM-3980) Performance tests are failing due to AttributeError: run_with_pdb

[ https://issues.apache.org/jira/browse/BEAM-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-3980. -- Resolution: Fixed Fix Version/s: Not applicable > Performance tests are failing

[jira] [Commented] (BEAM-3980) Performance tests are failing due to AttributeError: run_with_pdb

[ https://issues.apache.org/jira/browse/BEAM-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424199#comment-16424199 ] Chamikara Jayalath commented on BEAM-3980: -- Thanks. Closing this JIRA. > Performance tests are

[jira] [Created] (BEAM-3991) Update dependency 'google-api-services-storage' to latest version

Chamikara Jayalath created BEAM-3991: Summary: Update dependency 'google-api-services-storage' to latest version Key: BEAM-3991 URL: https://issues.apache.org/jira/browse/BEAM-3991 Project: Beam

[jira] [Commented] (BEAM-3990) Dataflow jobs fail with "KeyError: 'location'" when uploading to GCS

[ https://issues.apache.org/jira/browse/BEAM-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423403#comment-16423403 ] Chamikara Jayalath commented on BEAM-3990: -- Created rollback PR 

[jira] [Assigned] (BEAM-3990) Dataflow jobs fail with "KeyError: 'location'" when uploading to GCS

[ https://issues.apache.org/jira/browse/BEAM-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3990: Assignee: Charles Chen > Dataflow jobs fail with "KeyError: 'location'" when

[jira] [Created] (BEAM-3990) Dataflow jobs fail with "KeyError: 'location'" when uploading to GCS

Chamikara Jayalath created BEAM-3990: Summary: Dataflow jobs fail with "KeyError: 'location'" when uploading to GCS Key: BEAM-3990 URL: https://issues.apache.org/jira/browse/BEAM-3990 Project:

[jira] [Commented] (BEAM-3980) Performance tests are failing due to AttributeError: run_with_pdb

[ https://issues.apache.org/jira/browse/BEAM-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422663#comment-16422663 ] Chamikara Jayalath commented on BEAM-3980: -- cc: [~alanmyrvold]  [~ŁukaszG] > Performance tests

[jira] [Created] (BEAM-3980) Performance tests are failing due to AttributeError: run_with_pdb

Chamikara Jayalath created BEAM-3980: Summary: Performance tests are failing due to AttributeError: run_with_pdb Key: BEAM-3980 URL: https://issues.apache.org/jira/browse/BEAM-3980 Project: Beam

[jira] [Closed] (BEAM-3798) Performance tests flaky due to Dataflow transient errors

[ https://issues.apache.org/jira/browse/BEAM-3798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath closed BEAM-3798. Resolution: Fixed Fix Version/s: Not applicable > Performance tests flaky due to

[jira] [Commented] (BEAM-3798) Performance tests flaky due to Dataflow transient errors

[ https://issues.apache.org/jira/browse/BEAM-3798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418517#comment-16418517 ] Chamikara Jayalath commented on BEAM-3798: -- Sounds good. Closing. > Performance tests flaky due

[jira] [Resolved] (BEAM-3948) Quota exceeded error causes Performance tests failures.

[ https://issues.apache.org/jira/browse/BEAM-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-3948. -- Resolution: Fixed Fix Version/s: Not applicable > Quota exceeded error causes

[jira] [Commented] (BEAM-3948) Quota exceeded error causes Performance tests failures.

[ https://issues.apache.org/jira/browse/BEAM-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416626#comment-16416626 ] Chamikara Jayalath commented on BEAM-3948: -- Thanks Alan. Resolving this.  Let's reopen if quota

[jira] [Updated] (BEAM-3944) Convert beam_PerformanceTests_Python to use Gradle

[ https://issues.apache.org/jira/browse/BEAM-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3944: - Issue Type: Sub-task (was: Improvement) Parent: BEAM-3249 > Convert

[jira] [Updated] (BEAM-3942) Update performance testing framework to use Gradle.

[ https://issues.apache.org/jira/browse/BEAM-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3942: - Issue Type: Sub-task (was: Task) Parent: BEAM-3249 > Update performance testing

[jira] [Updated] (BEAM-3943) Convert existing Java performance tests to use Gradle

[ https://issues.apache.org/jira/browse/BEAM-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3943: - Issue Type: Sub-task (was: Improvement) Parent: BEAM-3249 > Convert existing Java

[jira] [Updated] (BEAM-3942) Update performance testing framework to use Gradle.

[ https://issues.apache.org/jira/browse/BEAM-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3942: - Issue Type: Task (was: Improvement) > Update performance testing framework to use Gradle.

[jira] [Commented] (BEAM-3942) Update performance testing framework to use Gradle.

[ https://issues.apache.org/jira/browse/BEAM-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414785#comment-16414785 ] Chamikara Jayalath commented on BEAM-3942: -- Assigning to Łukasz who has a PoC for this. > Update

[jira] [Assigned] (BEAM-3942) Update performance testing framework to use Gradle.

[ https://issues.apache.org/jira/browse/BEAM-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3942: Assignee: Łukasz Gajowy > Update performance testing framework to use Gradle. >

[jira] [Commented] (BEAM-3251) Migrate Performance Tests Jenkins Jobs to use Gradle

[ https://issues.apache.org/jira/browse/BEAM-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414783#comment-16414783 ] Chamikara Jayalath commented on BEAM-3251: -- Subtasks:

[jira] [Created] (BEAM-3944) Convert beam_PerformanceTests_Python to use Gradle

Chamikara Jayalath created BEAM-3944: Summary: Convert beam_PerformanceTests_Python to use Gradle Key: BEAM-3944 URL: https://issues.apache.org/jira/browse/BEAM-3944 Project: Beam Issue

[jira] [Created] (BEAM-3943) Convert existing Java performance tests to use Gradle

Chamikara Jayalath created BEAM-3943: Summary: Convert existing Java performance tests to use Gradle Key: BEAM-3943 URL: https://issues.apache.org/jira/browse/BEAM-3943 Project: Beam

[jira] [Created] (BEAM-3942) Update performance testing framework to use Gradle.

Chamikara Jayalath created BEAM-3942: Summary: Update performance testing framework to use Gradle. Key: BEAM-3942 URL: https://issues.apache.org/jira/browse/BEAM-3942 Project: Beam Issue

[jira] [Commented] (BEAM-3747) beam_PerformanceTests_JDBC is flaky

[ https://issues.apache.org/jira/browse/BEAM-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413440#comment-16413440 ] Chamikara Jayalath commented on BEAM-3747: -- Both JDBC and HadoopInputFormatIO tests are flaky due

[jira] [Updated] (BEAM-3923) Define and add updates needed to run a simple Java transform using Python SDK on top of the reference runner (ULR)

[ https://issues.apache.org/jira/browse/BEAM-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3923: - Component/s: sdk-py-core > Define and add updates needed to run a simple Java transform

[jira] [Assigned] (BEAM-3850) I/O transform for HDH5 files with extension H5

[ https://issues.apache.org/jira/browse/BEAM-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3850: Assignee: (was: Chamikara Jayalath) > I/O transform for HDH5 files with

[jira] [Assigned] (BEAM-3850) I/O transform for HDH5 files with extension H5

[ https://issues.apache.org/jira/browse/BEAM-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3850: Assignee: Chamikara Jayalath (was: Eugene Kirpichov) > I/O transform for HDH5

[jira] [Created] (BEAM-3923) Define and add updates needed to run a simple Java transform using Python SDK on top of the reference runner (ULR)

Chamikara Jayalath created BEAM-3923: Summary: Define and add updates needed to run a simple Java transform using Python SDK on top of the reference runner (ULR) Key: BEAM-3923 URL:

[jira] [Commented] (BEAM-2767) BigQueryIO result different for REPEATED field between DirectRunner and DataflowRunner

[ https://issues.apache.org/jira/browse/BEAM-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410634#comment-16410634 ] Chamikara Jayalath commented on BEAM-2767: -- Can you explain the difference between repeated values

[jira] [Assigned] (BEAM-2767) BigQueryIO result different for REPEATED field between DirectRunner and DataflowRunner

[ https://issues.apache.org/jira/browse/BEAM-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-2767: Assignee: Udi Meiri (was: Chamikara Jayalath) > BigQueryIO result different for

[jira] [Commented] (BEAM-3098) Upgrade Java grpc version

[ https://issues.apache.org/jira/browse/BEAM-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409015#comment-16409015 ] Chamikara Jayalath commented on BEAM-3098: -- Yeah, netty shaded gRPC is not available for 1.2.0 so

[jira] [Commented] (BEAM-3098) Upgrade Java grpc version

[ https://issues.apache.org/jira/browse/BEAM-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408963#comment-16408963 ] Chamikara Jayalath commented on BEAM-3098: -- BTW there is a gRPC version with netty shaded now: 

[jira] [Commented] (BEAM-3098) Upgrade Java grpc version

[ https://issues.apache.org/jira/browse/BEAM-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408952#comment-16408952 ] Chamikara Jayalath commented on BEAM-3098: -- This is something that has been in our radar but has

[jira] [Comment Edited] (BEAM-3098) Upgrade Java grpc version

[ https://issues.apache.org/jira/browse/BEAM-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406856#comment-16406856 ] Chamikara Jayalath edited comment on BEAM-3098 at 3/22/18 2:31 AM: ---

[jira] [Created] (BEAM-3907) Clarify how watermark is estimated for watchForNewFiles() transforms

Chamikara Jayalath created BEAM-3907: Summary: Clarify how watermark is estimated for watchForNewFiles() transforms Key: BEAM-3907 URL: https://issues.apache.org/jira/browse/BEAM-3907 Project:

[jira] [Commented] (BEAM-2810) Consider a faster Avro library in Python

[ https://issues.apache.org/jira/browse/BEAM-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407962#comment-16407962 ] Chamikara Jayalath commented on BEAM-2810: -- I only looked into fastavro. Basically, when reading

[jira] [Commented] (BEAM-3098) Upgrade Java grpc version

[ https://issues.apache.org/jira/browse/BEAM-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406856#comment-16406856 ] Chamikara Jayalath commented on BEAM-3098: -- DataflowRunner currently overrides and shards gRPC

[jira] [Assigned] (BEAM-3774) Update BigQuery jobs to explicitly specify the region

[ https://issues.apache.org/jira/browse/BEAM-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3774: Assignee: Chamikara Jayalath > Update BigQuery jobs to explicitly specify the

[jira] [Assigned] (BEAM-3867) Add support for decompressing and reading tar files

[ https://issues.apache.org/jira/browse/BEAM-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3867: Assignee: Sajeevan Achuthan > Add support for decompressing and reading tar files >

[jira] [Created] (BEAM-3867) Add support for decompressing and reading tar files

Chamikara Jayalath created BEAM-3867: Summary: Add support for decompressing and reading tar files Key: BEAM-3867 URL: https://issues.apache.org/jira/browse/BEAM-3867 Project: Beam Issue

[jira] [Commented] (BEAM-3824) BQ sink fails on Direct Runner for 2.4.0 RC2

[ https://issues.apache.org/jira/browse/BEAM-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393863#comment-16393863 ] Chamikara Jayalath commented on BEAM-3824: -- Thanks Charles. Above PR is being reverted. Assigning 

[jira] [Assigned] (BEAM-3824) BQ sink fails on Direct Runner for 2.4.0 RC2

[ https://issues.apache.org/jira/browse/BEAM-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3824: Assignee: David Cavazos (was: Chamikara Jayalath) > BQ sink fails on Direct Runner

[jira] [Commented] (BEAM-3824) BQ sink fails on Direct Runner for 2.4.0 RC2

[ https://issues.apache.org/jira/browse/BEAM-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393847#comment-16393847 ] Chamikara Jayalath commented on BEAM-3824: -- I confirmed that this works for 2.3 but doesn't work

[jira] [Commented] (BEAM-3824) BQ sink fails on Direct Runner for 2.4.0 RC2

[ https://issues.apache.org/jira/browse/BEAM-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393797#comment-16393797 ] Chamikara Jayalath commented on BEAM-3824: -- cc: [~charleschen] in case this is related to recent

[jira] [Commented] (BEAM-3824) BQ sink fails on Direct Runner for 2.4.0 RC2

[ https://issues.apache.org/jira/browse/BEAM-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393795#comment-16393795 ] Chamikara Jayalath commented on BEAM-3824: -- Did this work with Fn API runner before ? I've only

[jira] [Commented] (BEAM-3798) Performance tests flaky due to Dataflow transient errors

[ https://issues.apache.org/jira/browse/BEAM-3798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390022#comment-16390022 ] Chamikara Jayalath commented on BEAM-3798: -- Do you have the Dataflow job ID of a job that passes

[jira] [Created] (BEAM-3788) Implement a Kafka IO for Python SDK

Chamikara Jayalath created BEAM-3788: Summary: Implement a Kafka IO for Python SDK Key: BEAM-3788 URL: https://issues.apache.org/jira/browse/BEAM-3788 Project: Beam Issue Type: New

[jira] [Commented] (BEAM-3718) ClassNotFoundException on CloudResourceManager$Builder

[ https://issues.apache.org/jira/browse/BEAM-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388331#comment-16388331 ] Chamikara Jayalath commented on BEAM-3718: -- Looks like you somehow ended up picking an old version

[jira] [Created] (BEAM-3774) Update BigQuery jobs to explicitly specify the region

Chamikara Jayalath created BEAM-3774: Summary: Update BigQuery jobs to explicitly specify the region Key: BEAM-3774 URL: https://issues.apache.org/jira/browse/BEAM-3774 Project: Beam

[jira] [Assigned] (BEAM-3772) BigQueryIO - Can't use DynamicDestination with CREATE_IF_NEEDED for unbounded PCollection and FILE_LOADS

[ https://issues.apache.org/jira/browse/BEAM-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3772: Assignee: Eugene Kirpichov (was: Chamikara Jayalath) > BigQueryIO - Can't use

[jira] [Updated] (BEAM-3756) Update SpannerIO to use Batch API

[ https://issues.apache.org/jira/browse/BEAM-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3756: - Fix Version/s: (was: 2.4.0) Not applicable > Update SpannerIO to

[jira] [Commented] (BEAM-3289) Add ReadFromBigQuery and several other cleanups of bigquery.py

[ https://issues.apache.org/jira/browse/BEAM-3289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381235#comment-16381235 ] Chamikara Jayalath commented on BEAM-3289: -- "Make WriteToBigQuery to use Write(BigQuery) for batch

[jira] [Commented] (BEAM-3611) Split KafkaIO.java into smaller files

[ https://issues.apache.org/jira/browse/BEAM-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381160#comment-16381160 ] Chamikara Jayalath commented on BEAM-3611: -- Was this fixed by

[jira] [Commented] (BEAM-3689) Direct runner leak a reader for every 10 input records

[ https://issues.apache.org/jira/browse/BEAM-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381157#comment-16381157 ] Chamikara Jayalath commented on BEAM-3689: -- Looks like linked PR was merged. Can this be closed ?

[jira] [Updated] (BEAM-3519) GCP IO exposes netty on its API surface, causing conflicts with runners

[ https://issues.apache.org/jira/browse/BEAM-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3519: - Fix Version/s: (was: 2.4.0) > GCP IO exposes netty on its API surface, causing

[jira] [Updated] (BEAM-3756) Update SpannerIO to use Batch API

[ https://issues.apache.org/jira/browse/BEAM-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3756: - Affects Version/s: (was: 2.4.0) Not applicable > Update

[jira] [Updated] (BEAM-3756) Update SpannerIO to use Batch API

[ https://issues.apache.org/jira/browse/BEAM-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3756: - Fix Version/s: 2.4.0 > Update SpannerIO to use Batch API >

[jira] [Created] (BEAM-3756) Update SpannerIO to use Batch API

Chamikara Jayalath created BEAM-3756: Summary: Update SpannerIO to use Batch API Key: BEAM-3756 URL: https://issues.apache.org/jira/browse/BEAM-3756 Project: Beam Issue Type: Improvement

[jira] [Created] (BEAM-3747) beam_PerformanceTests_JDBC is flaky

Chamikara Jayalath created BEAM-3747: Summary: beam_PerformanceTests_JDBC is flaky Key: BEAM-3747 URL: https://issues.apache.org/jira/browse/BEAM-3747 Project: Beam Issue Type: Test

[jira] [Created] (BEAM-3719) Add side input support for SDFs running using streaming DirectRunner

Chamikara Jayalath created BEAM-3719: Summary: Add side input support for SDFs running using streaming DirectRunner Key: BEAM-3719 URL: https://issues.apache.org/jira/browse/BEAM-3719 Project:

[jira] [Commented] (BEAM-3519) GCP IO exposes netty on its API surface, causing conflicts with runners

[ https://issues.apache.org/jira/browse/BEAM-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357782#comment-16357782 ] Chamikara Jayalath commented on BEAM-3519: -- Were previous releases affected by this issue as well

[jira] [Commented] (BEAM-3519) GCP IO exposes netty on its API surface, causing conflicts with runners

[ https://issues.apache.org/jira/browse/BEAM-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357759#comment-16357759 ] Chamikara Jayalath commented on BEAM-3519: -- Possibly. Does this only have to be fixed for Maven

[jira] [Commented] (BEAM-3561) Provide kubernetes cluster instance for IOITs.

[ https://issues.apache.org/jira/browse/BEAM-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355855#comment-16355855 ] Chamikara Jayalath commented on BEAM-3561: -- This cluster is not used by any tests so should be

[jira] [Assigned] (BEAM-3561) Provide kubernetes cluster instance for IOITs.

[ https://issues.apache.org/jira/browse/BEAM-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3561: Assignee: Alan Myrvold (was: Jason Kuster) > Provide kubernetes cluster instance

[jira] [Resolved] (BEAM-3603) Add a ReadAll transform to tfrecordio

[ https://issues.apache.org/jira/browse/BEAM-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-3603. -- Resolution: Fixed Fix Version/s: 2.4.0 > Add a ReadAll transform to tfrecordio >

[jira] [Created] (BEAM-3603) Add a ReadAll transform to tfrecordio

Chamikara Jayalath created BEAM-3603: Summary: Add a ReadAll transform to tfrecordio Key: BEAM-3603 URL: https://issues.apache.org/jira/browse/BEAM-3603 Project: Beam Issue Type:

[jira] [Commented] (BEAM-3561) Provide kubernetes cluster instance for IOITs.

[ https://issues.apache.org/jira/browse/BEAM-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347805#comment-16347805 ] Chamikara Jayalath commented on BEAM-3561: -- Jenkins VMs already have necessary credentials to

[jira] [Assigned] (BEAM-601) Enable Kinesis integration tests

[ https://issues.apache.org/jira/browse/BEAM-601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-601: --- Assignee: Jason Kuster (was: Chamikara Jayalath) > Enable Kinesis integration tests >

[jira] [Assigned] (BEAM-601) Enable Kinesis integration tests

[ https://issues.apache.org/jira/browse/BEAM-601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-601: --- Assignee: (was: Jason Kuster) > Enable Kinesis integration tests >

[jira] [Commented] (BEAM-601) Enable Kinesis integration tests

[ https://issues.apache.org/jira/browse/BEAM-601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347586#comment-16347586 ] Chamikara Jayalath commented on BEAM-601: - I don't think we have access to AWS resources yet. It's

[jira] [Assigned] (BEAM-3557) Converting to runner api protos and back results in parent pointers being dropped

[ https://issues.apache.org/jira/browse/BEAM-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3557: Assignee: Chamikara Jayalath > Converting to runner api protos and back results in

[jira] [Resolved] (BEAM-1630) Add Splittable DoFn to Python SDK

[ https://issues.apache.org/jira/browse/BEAM-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-1630. -- Resolution: Fixed Fix Version/s: 2.3.0 > Add Splittable DoFn to Python SDK >

[jira] [Commented] (BEAM-1630) Add Splittable DoFn to Python SDK

[ https://issues.apache.org/jira/browse/BEAM-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344244#comment-16344244 ] Chamikara Jayalath commented on BEAM-1630: -- Closing this since the basic API and the

[jira] [Created] (BEAM-3567) Update DirectRunner's SDF implementation to support side outputs

Chamikara Jayalath created BEAM-3567: Summary: Update DirectRunner's SDF implementation to support side outputs Key: BEAM-3567 URL: https://issues.apache.org/jira/browse/BEAM-3567 Project: Beam

[jira] [Commented] (BEAM-3480) Jenkins IOIT jobs unstable

[ https://issues.apache.org/jira/browse/BEAM-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344125#comment-16344125 ] Chamikara Jayalath commented on BEAM-3480: -- Alan pointed out following flag.

[jira] [Commented] (BEAM-3557) Converting to runner api protos and back results in parent pointers being dropped

[ https://issues.apache.org/jira/browse/BEAM-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342831#comment-16342831 ] Chamikara Jayalath commented on BEAM-3557: -- cc: [~charleschen] [~robertwb] > Converting to runner

[jira] [Created] (BEAM-3557) Converting to runner api protos and back results in parent pointers being dropped

Chamikara Jayalath created BEAM-3557: Summary: Converting to runner api protos and back results in parent pointers being dropped Key: BEAM-3557 URL: https://issues.apache.org/jira/browse/BEAM-3557

[jira] [Assigned] (BEAM-3501) BigQuery Partitioned table creation/write fails when destination has partition decorator

[ https://issues.apache.org/jira/browse/BEAM-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3501: Assignee: Eugene Kirpichov (was: Chamikara Jayalath) > BigQuery Partitioned table

[jira] [Commented] (BEAM-3433) Allow BigQueryIO to use a different project for the load job in batch mode.

[ https://issues.apache.org/jira/browse/BEAM-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339660#comment-16339660 ] Chamikara Jayalath commented on BEAM-3433: -- Thanks. I agree that this is a valid use-case and

[jira] [Commented] (BEAM-3441) Allow ValueProvider for JdbcIO.DataSourceConfiguration

[ https://issues.apache.org/jira/browse/BEAM-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336473#comment-16336473 ] Chamikara Jayalath commented on BEAM-3441: -- Thanks. I'll take a look.     > Allow ValueProvider

[jira] [Commented] (BEAM-3412) Update BigTable client version to 1.0

[ https://issues.apache.org/jira/browse/BEAM-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334746#comment-16334746 ] Chamikara Jayalath commented on BEAM-3412: -- Thanks. Will take a look. > Update BigTable client

<    1   2   3   4   5   >