[jira] [Updated] (BEAM-7063) beam_PostCommit_Python_Verify is consistently failing due to a sideinput materialization related issue

[ https://issues.apache.org/jira/browse/BEAM-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-7063: - Issue Type: Bug (was: Improvement) > beam_PostCommit_Python_Verify is consistently

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815863#comment-16815863 ] Chamikara Jayalath commented on BEAM-2572: -- Note that this involves implementing following

[jira] [Updated] (BEAM-7063) beam_PostCommit_Python_Verify is consistently failing due to a sideinput materialization related issue

[ https://issues.apache.org/jira/browse/BEAM-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-7063: - Description: Seems like this was the first post-commit failure: 

[jira] [Created] (BEAM-7063) beam_PostCommit_Python_Verify is consistently failing due to a sideinput materialization related issue

Chamikara Jayalath created BEAM-7063: Summary: beam_PostCommit_Python_Verify is consistently failing due to a sideinput materialization related issue Key: BEAM-7063 URL:

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815748#comment-16815748 ] Chamikara Jayalath commented on BEAM-2572: -- See here for some context regarding Summer of code

[jira] [Commented] (BEAM-6590) LTS backport: upgrade gcsio dependency to 1.9.13

[ https://issues.apache.org/jira/browse/BEAM-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808946#comment-16808946 ] Chamikara Jayalath commented on BEAM-6590: -- Releasing for now since I'm busy due to other things.

[jira] [Assigned] (BEAM-6590) LTS backport: upgrade gcsio dependency to 1.9.13

[ https://issues.apache.org/jira/browse/BEAM-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-6590: Assignee: (was: Chamikara Jayalath) > LTS backport: upgrade gcsio dependency

[jira] [Created] (BEAM-6962) Add wordcount_xlang to Python post-commit

Chamikara Jayalath created BEAM-6962: Summary: Add wordcount_xlang to Python post-commit Key: BEAM-6962 URL: https://issues.apache.org/jira/browse/BEAM-6962 Project: Beam Issue Type:

[jira] [Assigned] (BEAM-6894) ExternalTransform.expand() does not create the proper AppliedPTransform sub-graph

[ https://issues.apache.org/jira/browse/BEAM-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-6894: Assignee: Chamikara Jayalath > ExternalTransform.expand() does not create the

[jira] [Commented] (BEAM-6707) TextIO.Write appear success but request not sent to Google Cloud Storage

[ https://issues.apache.org/jira/browse/BEAM-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802047#comment-16802047 ] Chamikara Jayalath commented on BEAM-6707: -- Why would FileSystems.rename(srcFiles, dstFiles,

[jira] [Commented] (BEAM-6894) ExternalTransform.expand() does not create the proper AppliedPTransform sub-graph

[ https://issues.apache.org/jira/browse/BEAM-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799670#comment-16799670 ] Chamikara Jayalath commented on BEAM-6894: -- [~robertwb], do you have any ideas on how to properly

[jira] [Created] (BEAM-6894) ExternalTransform.expand() does not create the proper AppliedPTransform sub-graph

Chamikara Jayalath created BEAM-6894: Summary: ExternalTransform.expand() does not create the proper AppliedPTransform sub-graph Key: BEAM-6894 URL: https://issues.apache.org/jira/browse/BEAM-6894

[jira] [Created] (BEAM-6852) Produce a better error message when GCS bucket is not available when writing

Chamikara Jayalath created BEAM-6852: Summary: Produce a better error message when GCS bucket is not available when writing Key: BEAM-6852 URL: https://issues.apache.org/jira/browse/BEAM-6852

[jira] [Updated] (BEAM-6852) Produce a better error message when GCS bucket is not available when writing

[ https://issues.apache.org/jira/browse/BEAM-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6852: - Issue Type: Improvement (was: Test) > Produce a better error message when GCS bucket is

[jira] [Commented] (BEAM-6769) BigQuery IO does not support bytes in Python 3

[ https://issues.apache.org/jira/browse/BEAM-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793809#comment-16793809 ] Chamikara Jayalath commented on BEAM-6769: -- "Why should unicode values be base64 encoded?" we

[jira] [Commented] (BEAM-6769) BigQuery IO does not support bytes in Python 3

[ https://issues.apache.org/jira/browse/BEAM-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793816#comment-16793816 ] Chamikara Jayalath commented on BEAM-6769: -- Also, please summarize this discussion in the dev

[jira] [Commented] (BEAM-6769) BigQuery IO does not support bytes in Python 3

[ https://issues.apache.org/jira/browse/BEAM-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793232#comment-16793232 ] Chamikara Jayalath commented on BEAM-6769: -- Thanks for the comments and the PR. Looks like, for

[jira] [Commented] (BEAM-6683) Add an integration test suite for cross-language transforms for Flink runner

[ https://issues.apache.org/jira/browse/BEAM-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788434#comment-16788434 ] Chamikara Jayalath commented on BEAM-6683: -- I agree that Schemas probably should be the data

[jira] [Commented] (BEAM-6683) Add an integration test suite for cross-language transforms for Flink runner

[ https://issues.apache.org/jira/browse/BEAM-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788357#comment-16788357 ] Chamikara Jayalath commented on BEAM-6683: -- Possibly we should target developing some sort of a

[jira] [Commented] (BEAM-6683) Add an integration test suite for cross-language transforms for Flink runner

[ https://issues.apache.org/jira/browse/BEAM-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788355#comment-16788355 ] Chamikara Jayalath commented on BEAM-6683: -- Heejong, could you please update the status of this

[jira] [Commented] (BEAM-6697) ParquetIO Performance test is failing on (GCS filesystem)

[ https://issues.apache.org/jira/browse/BEAM-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777499#comment-16777499 ] Chamikara Jayalath commented on BEAM-6697: -- I confirmed that the test passes with the library

[jira] [Commented] (BEAM-6697) ParquetIO Performance test is failing on (GCS filesystem)

[ https://issues.apache.org/jira/browse/BEAM-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777199#comment-16777199 ] Chamikara Jayalath commented on BEAM-6697: -- [~medb] (owner of gcsio library) is taking a look. >

[jira] [Commented] (BEAM-6697) ParquetIO Performance test is failing on (GCS filesystem)

[ https://issues.apache.org/jira/browse/BEAM-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777177#comment-16777177 ] Chamikara Jayalath commented on BEAM-6697: -- Looks like it's from here:

[jira] [Resolved] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-6552. -- Resolution: Fixed > Upgrade bigdataoss_gcsio dependency >

[jira] [Updated] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6552: - Fix Version/s: (was: 2.7.1) 2.10.0 > Upgrade bigdataoss_gcsio

[jira] [Commented] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762051#comment-16762051 ] Chamikara Jayalath commented on BEAM-6552: -- Closing this. Created

[jira] [Created] (BEAM-6618) Upgrade gcsio version to 1.9.13

Chamikara Jayalath created BEAM-6618: Summary: Upgrade gcsio version to 1.9.13 Key: BEAM-6618 URL: https://issues.apache.org/jira/browse/BEAM-6618 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-6582) Upgrade gcsio dependency to 1.9.13

[ https://issues.apache.org/jira/browse/BEAM-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762050#comment-16762050 ] Chamikara Jayalath commented on BEAM-6582: -- Ok. Created

[jira] [Updated] (BEAM-6582) Upgrade gcsio dependency to 1.9.13

[ https://issues.apache.org/jira/browse/BEAM-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6582: - Fix Version/s: (was: 2.10.0) > Upgrade gcsio dependency to 1.9.13 >

[jira] [Commented] (BEAM-6064) Python BigQuery performance much worse than Java

[ https://issues.apache.org/jira/browse/BEAM-6064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760247#comment-16760247 ] Chamikara Jayalath commented on BEAM-6064: -- Do you have a Dataflow Job ID for a run with fastavro

[jira] [Updated] (BEAM-6582) Upgrade gcsio dependency to 1.9.13

[ https://issues.apache.org/jira/browse/BEAM-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6582: - Affects Version/s: (was: 2.10.0) > Upgrade gcsio dependency to 1.9.13 >

[jira] [Updated] (BEAM-6582) Upgrade gcsio dependency to 1.9.13

[ https://issues.apache.org/jira/browse/BEAM-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6582: - Fix Version/s: 2.10.0 2.7.1 > Upgrade gcsio dependency to 1.9.13 >

[jira] [Updated] (BEAM-6582) Upgrade gcsio dependency to 1.9.13

[ https://issues.apache.org/jira/browse/BEAM-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6582: - Affects Version/s: (was: 2.7.1) > Upgrade gcsio dependency to 1.9.13 >

[jira] [Created] (BEAM-6582) Upgrade gcsio dependency to 1.9.13

Chamikara Jayalath created BEAM-6582: Summary: Upgrade gcsio dependency to 1.9.13 Key: BEAM-6582 URL: https://issues.apache.org/jira/browse/BEAM-6582 Project: Beam Issue Type: Bug

[jira] [Updated] (BEAM-6581) beam_PostRelease_NightlySnapshot is failing in node beam11 due to a SocketException

[ https://issues.apache.org/jira/browse/BEAM-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6581: - Description: For example,

[jira] [Created] (BEAM-6581) beam_PostRelease_NightlySnapshot is failing in node beam11 due to a SocketException

Chamikara Jayalath created BEAM-6581: Summary: beam_PostRelease_NightlySnapshot is failing in node beam11 due to a SocketException Key: BEAM-6581 URL: https://issues.apache.org/jira/browse/BEAM-6581

[jira] [Commented] (BEAM-4822) Beam FileIO should support versioned file systems

[ https://issues.apache.org/jira/browse/BEAM-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758551#comment-16758551 ] Chamikara Jayalath commented on BEAM-4822: -- I think one of the issues is updates to current

[jira] [Updated] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6552: - Fix Version/s: (was: 2.10.0) > Upgrade bigdataoss_gcsio dependency >

[jira] [Updated] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6552: - Priority: Major (was: Blocker) > Upgrade bigdataoss_gcsio dependency >

[jira] [Commented] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757721#comment-16757721 ] Chamikara Jayalath commented on BEAM-6552: -- PRs were merged to HEAD and 2.10.0 branch. So I'll

[jira] [Updated] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6552: - Fix Version/s: 2.7.1 > Upgrade bigdataoss_gcsio dependency >

[jira] [Comment Edited] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756611#comment-16756611 ] Chamikara Jayalath edited comment on BEAM-6552 at 1/31/19 1:51 AM: --- I

[jira] [Comment Edited] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756611#comment-16756611 ] Chamikara Jayalath edited comment on BEAM-6552 at 1/31/19 1:46 AM: --- I

[jira] [Comment Edited] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756611#comment-16756611 ] Chamikara Jayalath edited comment on BEAM-6552 at 1/30/19 10:15 PM: I

[jira] [Commented] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756611#comment-16756611 ] Chamikara Jayalath commented on BEAM-6552: -- I believe this can potentially affect all released

[jira] [Updated] (BEAM-4822) Beam FileIO should support versioned file systems

[ https://issues.apache.org/jira/browse/BEAM-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-4822: - Issue Type: New Feature (was: Bug) > Beam FileIO should support versioned file systems >

[jira] [Commented] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

[ https://issues.apache.org/jira/browse/BEAM-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756384#comment-16756384 ] Chamikara Jayalath commented on BEAM-6552: -- We'll have to upgrade to bigdataoss_gcsio 1.12.0

[jira] [Created] (BEAM-6552) Upgrade bigdataoss_gcsio dependency

Chamikara Jayalath created BEAM-6552: Summary: Upgrade bigdataoss_gcsio dependency Key: BEAM-6552 URL: https://issues.apache.org/jira/browse/BEAM-6552 Project: Beam Issue Type:

[jira] [Created] (BEAM-6543) Multiple PostCommit_Java_ValidatesRunner_Flink tests are failing due to AssertionError

Chamikara Jayalath created BEAM-6543: Summary: Multiple PostCommit_Java_ValidatesRunner_Flink tests are failing due to AssertionError Key: BEAM-6543 URL: https://issues.apache.org/jira/browse/BEAM-6543

[jira] [Assigned] (BEAM-6539) Some Java PostCommits failing: Coder casting error.

[ https://issues.apache.org/jira/browse/BEAM-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-6539: Assignee: Robert Bradshaw (was: Daniel Oliveira) > Some Java PostCommits failing:

[jira] [Resolved] (BEAM-4620) UnboundedReadFromBoundedSource.split() should always call split()

[ https://issues.apache.org/jira/browse/BEAM-4620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-4620. -- Resolution: Fixed Fix Version/s: 2.11.0 > UnboundedReadFromBoundedSource.split()

[jira] [Created] (BEAM-6486) Cross-language transform API

Chamikara Jayalath created BEAM-6486: Summary: Cross-language transform API Key: BEAM-6486 URL: https://issues.apache.org/jira/browse/BEAM-6486 Project: Beam Issue Type: New Feature

[jira] [Updated] (BEAM-6488) Portable Flink runner support for running cross-language transforms

[ https://issues.apache.org/jira/browse/BEAM-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6488: - Description: To support running cross-language transforms, Portable Flink runner needs to

[jira] [Created] (BEAM-6488) Flink runner support for running cross-language transforms

Chamikara Jayalath created BEAM-6488: Summary: Flink runner support for running cross-language transforms Key: BEAM-6488 URL: https://issues.apache.org/jira/browse/BEAM-6488 Project: Beam

[jira] [Updated] (BEAM-6488) Portable Flink runner support for running cross-language transforms

[ https://issues.apache.org/jira/browse/BEAM-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6488: - Summary: Portable Flink runner support for running cross-language transforms (was: Flink

[jira] [Created] (BEAM-6485) Cross-language transform expansion protocol

Chamikara Jayalath created BEAM-6485: Summary: Cross-language transform expansion protocol Key: BEAM-6485 URL: https://issues.apache.org/jira/browse/BEAM-6485 Project: Beam Issue Type:

[jira] [Commented] (BEAM-6391) fix miscellaneous bugs

[ https://issues.apache.org/jira/browse/BEAM-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742571#comment-16742571 ] Chamikara Jayalath commented on BEAM-6391: -- Please give a more descriptive name (sounds like a

[jira] [Assigned] (BEAM-5614) Using gs:// paths without first doing a "gcloud auth" gives an unhelpful error message

[ https://issues.apache.org/jira/browse/BEAM-5614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-5614: Assignee: (was: Chamikara Jayalath) > Using gs:// paths without first doing a

[jira] [Assigned] (BEAM-5420) BigtableIO tries to get runtime parameters when collecting display data at pipeline construction time

[ https://issues.apache.org/jira/browse/BEAM-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-5420: Assignee: Pablo Estrada (was: Chamikara Jayalath) > BigtableIO tries to get

[jira] [Assigned] (BEAM-4904) Beam Dependency Update Request: de.flapdoodle.embed:de.flapdoodle.embed.mongo 2.1.1

[ https://issues.apache.org/jira/browse/BEAM-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-4904: Assignee: (was: Chamikara Jayalath) > Beam Dependency Update Request:

[jira] [Commented] (BEAM-5417) FileSystems.match behaviour diff between GCS and local file system

[ https://issues.apache.org/jira/browse/BEAM-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739591#comment-16739591 ] Chamikara Jayalath commented on BEAM-5417: -- Joar, did you try with the latest version of Beam ?

[jira] [Assigned] (BEAM-5934) FileSink affected by S3 eventual consistency

[ https://issues.apache.org/jira/browse/BEAM-5934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-5934: Assignee: (was: Chamikara Jayalath) > FileSink affected by S3 eventual

[jira] [Assigned] (BEAM-5676) Allow to change the PubsubClientFactory when using PubsubIO

[ https://issues.apache.org/jira/browse/BEAM-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-5676: Assignee: (was: Chamikara Jayalath) > Allow to change the PubsubClientFactory

[jira] [Assigned] (BEAM-5417) FileSystems.match behaviour diff between GCS and local file system

[ https://issues.apache.org/jira/browse/BEAM-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-5417: Assignee: Udi Meiri (was: Chamikara Jayalath) > FileSystems.match behaviour diff

[jira] [Commented] (BEAM-5676) Allow to change the PubsubClientFactory when using PubsubIO

[ https://issues.apache.org/jira/browse/BEAM-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739589#comment-16739589 ] Chamikara Jayalath commented on BEAM-5676: -- Logan, please feel free to send a pull request if you

[jira] [Comment Edited] (BEAM-5694) OOMs on Pub/Sub to BigQuery via FILE_LOADS

[ https://issues.apache.org/jira/browse/BEAM-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739588#comment-16739588 ] Chamikara Jayalath edited comment on BEAM-5694 at 1/10/19 4:31 PM: --- CC:

[jira] [Assigned] (BEAM-5694) OOMs on Pub/Sub to BigQuery via FILE_LOADS

[ https://issues.apache.org/jira/browse/BEAM-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-5694: Assignee: (was: Chamikara Jayalath) > OOMs on Pub/Sub to BigQuery via

[jira] [Commented] (BEAM-5694) OOMs on Pub/Sub to BigQuery via FILE_LOADS

[ https://issues.apache.org/jira/browse/BEAM-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739588#comment-16739588 ] Chamikara Jayalath commented on BEAM-5694: -- R: [~reuvenlax] > OOMs on Pub/Sub to BigQuery via

[jira] [Assigned] (BEAM-5929) BigQueryIO DynamicDestination should support JSONSchema

[ https://issues.apache.org/jira/browse/BEAM-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-5929: Assignee: (was: Chamikara Jayalath) > BigQueryIO DynamicDestination should

[jira] [Assigned] (BEAM-5635) FR: Enable Transactional writes with DatastoreIO

[ https://issues.apache.org/jira/browse/BEAM-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-5635: Assignee: (was: Chamikara Jayalath) > FR: Enable Transactional writes with

[jira] [Commented] (BEAM-5966) When working "LiteralGqlQuery", Error: Query cannot have any sort orders.

[ https://issues.apache.org/jira/browse/BEAM-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739585#comment-16739585 ] Chamikara Jayalath commented on BEAM-5966: -- Was your question resolved or is there something more

[jira] [Assigned] (BEAM-6061) When using unbounded PCollection from TextIO to BigQuery, data is stuck in Reshuffle/GroupByKey inside of BigQueryIO

[ https://issues.apache.org/jira/browse/BEAM-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-6061: Assignee: (was: Chamikara Jayalath) > When using unbounded PCollection from

[jira] [Resolved] (BEAM-6064) Python BigQuery performance much worse than Java

[ https://issues.apache.org/jira/browse/BEAM-6064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-6064. -- Resolution: Fixed Fix Version/s: 2.9.0 > Python BigQuery performance much worse

[jira] [Commented] (BEAM-6064) Python BigQuery performance much worse than Java

[ https://issues.apache.org/jira/browse/BEAM-6064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739582#comment-16739582 ] Chamikara Jayalath commented on BEAM-6064: -- Fastavro support for BigQuery for DataflowRunner is

[jira] [Commented] (BEAM-6169) Dataflow runner: rateLimit exceeded from BigQuery API in start_bundle

[ https://issues.apache.org/jira/browse/BEAM-6169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739577#comment-16739577 ] Chamikara Jayalath commented on BEAM-6169: -- Did your pipeline stall due to this ? Otherwise

[jira] [Assigned] (BEAM-6206) Dataflow template which reads from BigQuery fails if used more than once

[ https://issues.apache.org/jira/browse/BEAM-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-6206: Assignee: Chamikara Jayalath (was: Tyler Akidau) > Dataflow template which reads

[jira] [Commented] (BEAM-4520) No messages delivered after a while with PubsubIO

[ https://issues.apache.org/jira/browse/BEAM-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739570#comment-16739570 ] Chamikara Jayalath commented on BEAM-4520: -- cc: [~reuvenlax] [~mil...@google.com] > No messages

[jira] [Commented] (BEAM-5806) Allow to change the PubsubClientFactory when using PubsubIO

[ https://issues.apache.org/jira/browse/BEAM-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739571#comment-16739571 ] Chamikara Jayalath commented on BEAM-5806: -- cc: [~reuvenlax] [~mil...@google.com] > Allow to

[jira] [Assigned] (BEAM-5806) Allow to change the PubsubClientFactory when using PubsubIO

[ https://issues.apache.org/jira/browse/BEAM-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-5806: Assignee: (was: Chamikara Jayalath) > Allow to change the PubsubClientFactory

[jira] [Updated] (BEAM-6206) Dataflow template which reads from BigQuery fails if used more than once

[ https://issues.apache.org/jira/browse/BEAM-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6206: - Priority: Major (was: Blocker) > Dataflow template which reads from BigQuery fails if

[jira] [Assigned] (BEAM-4520) No messages delivered after a while with PubsubIO

[ https://issues.apache.org/jira/browse/BEAM-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-4520: Assignee: (was: Chamikara Jayalath) > No messages delivered after a while with

[jira] [Commented] (BEAM-6206) Dataflow template which reads from BigQuery fails if used more than once

[ https://issues.apache.org/jira/browse/BEAM-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739568#comment-16739568 ] Chamikara Jayalath commented on BEAM-6206: -- You have to use withTemplateCompatibility() for

[jira] [Resolved] (BEAM-6206) Dataflow template which reads from BigQuery fails if used more than once

[ https://issues.apache.org/jira/browse/BEAM-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-6206. -- Resolution: Fixed Fix Version/s: Not applicable > Dataflow template which reads

[jira] [Comment Edited] (BEAM-5434) Issue with BigQueryIO in Template

[ https://issues.apache.org/jira/browse/BEAM-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739564#comment-16739564 ] Chamikara Jayalath edited comment on BEAM-5434 at 1/10/19 4:14 PM: --- You

[jira] [Updated] (BEAM-5434) Issue with BigQueryIO in Template

[ https://issues.apache.org/jira/browse/BEAM-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-5434: - Priority: Major (was: Blocker) > Issue with BigQueryIO in Template >

[jira] [Resolved] (BEAM-5434) Issue with BigQueryIO in Template

[ https://issues.apache.org/jira/browse/BEAM-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-5434. -- Resolution: Fixed Fix Version/s: Not applicable > Issue with BigQueryIO in

[jira] [Commented] (BEAM-5434) Issue with BigQueryIO in Template

[ https://issues.apache.org/jira/browse/BEAM-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739564#comment-16739564 ] Chamikara Jayalath commented on BEAM-5434: -- You have to use withTemplateCompatiblity() for

[jira] [Assigned] (BEAM-6320) SpannerReadIT.testQuery flaky

[ https://issues.apache.org/jira/browse/BEAM-6320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-6320: Assignee: (was: Chamikara Jayalath) > SpannerReadIT.testQuery flaky >

[jira] [Assigned] (BEAM-6337) testE2EV1ReadWithGQLQueryWithNoLimit failed owing to assertion error

[ https://issues.apache.org/jira/browse/BEAM-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-6337: Assignee: (was: Chamikara Jayalath) > testE2EV1ReadWithGQLQueryWithNoLimit

[jira] [Commented] (BEAM-6344) :beam-sdks-java-io-elasticsearch-tests-2:test failed owing to get 503 from ES

[ https://issues.apache.org/jira/browse/BEAM-6344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739560#comment-16739560 ] Chamikara Jayalath commented on BEAM-6344: -- cc: [~echauchot] [~ŁukaszG] >

[jira] [Assigned] (BEAM-6399) FileIO errors on unbounded input with nondefault trigger

[ https://issues.apache.org/jira/browse/BEAM-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-6399: Assignee: (was: Eugene Kirpichov) > FileIO errors on unbounded input with

[jira] [Updated] (BEAM-3519) GCP IO exposes netty on its API surface, causing conflicts with runners

[ https://issues.apache.org/jira/browse/BEAM-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-3519: - Priority: Major (was: Critical) > GCP IO exposes netty on its API surface, causing

[jira] [Commented] (BEAM-3519) GCP IO exposes netty on its API surface, causing conflicts with runners

[ https://issues.apache.org/jira/browse/BEAM-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738453#comment-16738453 ] Chamikara Jayalath commented on BEAM-3519: -- I'll reduce the priority and pass to Kenn since he

[jira] [Assigned] (BEAM-3519) GCP IO exposes netty on its API surface, causing conflicts with runners

[ https://issues.apache.org/jira/browse/BEAM-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-3519: Assignee: Kenneth Knowles (was: Chamikara Jayalath) > GCP IO exposes netty on its

[jira] [Commented] (BEAM-3519) GCP IO exposes netty on its API surface, causing conflicts with runners

[ https://issues.apache.org/jira/browse/BEAM-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738451#comment-16738451 ] Chamikara Jayalath commented on BEAM-3519: -- We have upgraded gRPC and netty dependencies used by

[jira] [Assigned] (BEAM-6392) Add support for new BigQuery streaming read API to BigQueryIO

[ https://issues.apache.org/jira/browse/BEAM-6392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-6392: Assignee: Kenneth Jung (was: Chamikara Jayalath) > Add support for new BigQuery

[jira] [Resolved] (BEAM-6106) test_bigquery_tornadoes_it fails due to a hash mismatch

[ https://issues.apache.org/jira/browse/BEAM-6106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-6106. -- Resolution: Fixed Fix Version/s: Not applicable > test_bigquery_tornadoes_it

[jira] [Commented] (BEAM-6312) org.apache.beam.sdk.io.gcp.bigquery.BigQueryToTableIT.testNewTypesQueryWithoutReshuffle flakey

[ https://issues.apache.org/jira/browse/BEAM-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729213#comment-16729213 ] Chamikara Jayalath commented on BEAM-6312: -- Please note that some of these flaky tests are new.

[jira] [Commented] (BEAM-6312) org.apache.beam.sdk.io.gcp.bigquery.BigQueryToTableIT.testNewTypesQueryWithoutReshuffle flakey

[ https://issues.apache.org/jira/browse/BEAM-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729214#comment-16729214 ] Chamikara Jayalath commented on BEAM-6312: -- (unless you suspect an actual issue in GCP IO) > >

[jira] [Commented] (BEAM-6249) Vendored gRPC doesn't seem to work with dataflow

[ https://issues.apache.org/jira/browse/BEAM-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726335#comment-16726335 ] Chamikara Jayalath commented on BEAM-6249: -- We could reproduce this issue and looking for a

<    1   2   3   >