[GitHub] [beam] codecov[bot] edited a comment on pull request #12867: Pub/Sub Lite getSplitBacklog implementation

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12867: URL: https://github.com/apache/beam/pull/12867#issuecomment-698115944 # [Codecov](https://codecov.io/gh/apache/beam/pull/12867?src=pr=h1) Report > Merging [#12867](https://codecov.io/gh/apache/beam/pull/12867?src=pr=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12867: Pub/Sub Lite getSplitBacklog implementation

2020-09-23 Thread GitBox
codecov[bot] commented on pull request #12867: URL: https://github.com/apache/beam/pull/12867#issuecomment-698115944 # [Codecov](https://codecov.io/gh/apache/beam/pull/12867?src=pr=h1) Report > Merging [#12867](https://codecov.io/gh/apache/beam/pull/12867?src=pr=desc) into

[GitHub] [beam] vmarquez commented on pull request #10546: [BEAM-9008] Add CassandraIO readAll method

2020-09-23 Thread GitBox
vmarquez commented on pull request #10546: URL: https://github.com/apache/beam/pull/10546#issuecomment-698100961 @iemejia Apologies for the long delay, I finally had time to deal with the big merge and I believe I have addressed all the issues. Maybe a few nits I missed. I also realized

[GitHub] [beam] stale[bot] commented on pull request #10546: [BEAM-9008] Add CassandraIO readAll method

2020-09-23 Thread GitBox
stale[bot] commented on pull request #10546: URL: https://github.com/apache/beam/pull/10546#issuecomment-698079325 This pull request is no longer marked as stale. This is an automated message from the Apache Git Service.

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr=desc) into

[GitHub] [beam] boyuanzz opened a new pull request #12926: [WIP] Clean up with_attributes from PubsubSource

2020-09-23 Thread GitBox
boyuanzz opened a new pull request #12926: URL: https://github.com/apache/beam/pull/12926 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr=desc) into

[GitHub] [beam] lostluck merged pull request #12925: [BEAM-9616] Consolidate Element and DoFn json impl

2020-09-23 Thread GitBox
lostluck merged pull request #12925: URL: https://github.com/apache/beam/pull/12925 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr=desc) into

[GitHub] [beam] tvalentyn closed pull request #12645: [BEAM-10124] Add ContextualTextIO

2020-09-23 Thread GitBox
tvalentyn closed pull request #12645: URL: https://github.com/apache/beam/pull/12645 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] tvalentyn commented on pull request #12645: [BEAM-10124] Add ContextualTextIO

2020-09-23 Thread GitBox
tvalentyn commented on pull request #12645: URL: https://github.com/apache/beam/pull/12645#issuecomment-698059215 Let's continue the discussion on #12924 which includes commits from https://github.com/apache/beam/pull/12645 and https://github.com/abhiy13/beam/pull/2.

[GitHub] [beam] tiny-dancer commented on pull request #12282: [WIP][BEAM-7819] Python - Include message_id and publish_time in PubsubMessage

2020-09-23 Thread GitBox
tiny-dancer commented on pull request #12282: URL: https://github.com/apache/beam/pull/12282#issuecomment-698056850 @udim no longer being worked on, last left off where it was working locally but not in dataflow. Feel free to proceed, thank you!

[GitHub] [beam] lostluck commented on pull request #12925: [BEAM-9616] Consolidate Element and DoFn json impl

2020-09-23 Thread GitBox
lostluck commented on pull request #12925: URL: https://github.com/apache/beam/pull/12925#issuecomment-698054407 cc: @dsnet This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] lostluck commented on pull request #12925: [BEAM-9616] Consolidate Element and DoFn json impl

2020-09-23 Thread GitBox
lostluck commented on pull request #12925: URL: https://github.com/apache/beam/pull/12925#issuecomment-698052466 R: @youngoli This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] lostluck opened a new pull request #12925: [BEAM-9616] Consolidate Element and DoFn json impl

2020-09-23 Thread GitBox
lostluck opened a new pull request #12925: URL: https://github.com/apache/beam/pull/12925 Consolidating the implementations of json encoding to a jsonx package, which will make it easier to replace with Schemas later. Thank you for your contribution!

[GitHub] [beam] udim commented on pull request #12282: [WIP][BEAM-7819] Python - Include message_id and publish_time in PubsubMessage

2020-09-23 Thread GitBox
udim commented on pull request #12282: URL: https://github.com/apache/beam/pull/12282#issuecomment-698051439 Hi, is this still being worked on? I'm asking because I might pick this up. If you want to work on this please let me know and I'd be happy to answer any questions.

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: Contextualio rebase

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12923: Add to_pcollection example to wordcount_dataframe

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12923: URL: https://github.com/apache/beam/pull/12923#issuecomment-698040203 # [Codecov](https://codecov.io/gh/apache/beam/pull/12923?src=pr=h1) Report > Merging [#12923](https://codecov.io/gh/apache/beam/pull/12923?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: Contextualio rebase

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12923: Add to_pcollection example to wordcount_dataframe

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12923: URL: https://github.com/apache/beam/pull/12923#issuecomment-698040203 # [Codecov](https://codecov.io/gh/apache/beam/pull/12923?src=pr=h1) Report > Merging [#12923](https://codecov.io/gh/apache/beam/pull/12923?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12923: Add to_pcollection example to wordcount_dataframe

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12923: URL: https://github.com/apache/beam/pull/12923#issuecomment-698040203 # [Codecov](https://codecov.io/gh/apache/beam/pull/12923?src=pr=h1) Report > Merging [#12923](https://codecov.io/gh/apache/beam/pull/12923?src=pr=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12924: Contextualio rebase

2020-09-23 Thread GitBox
codecov[bot] commented on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12923: Add to_pcollection example to wordcount_dataframe

2020-09-23 Thread GitBox
codecov[bot] commented on pull request #12923: URL: https://github.com/apache/beam/pull/12923#issuecomment-698040203 # [Codecov](https://codecov.io/gh/apache/beam/pull/12923?src=pr=h1) Report > Merging [#12923](https://codecov.io/gh/apache/beam/pull/12923?src=pr=desc) into

[GitHub] [beam] ibzib merged pull request #12905: [BEAM-8660] Use PortableOptions.artifact_endpoint if provided over value reported from jobserver

2020-09-23 Thread GitBox
ibzib merged pull request #12905: URL: https://github.com/apache/beam/pull/12905 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] tvalentyn merged pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-23 Thread GitBox
tvalentyn merged pull request #12727: URL: https://github.com/apache/beam/pull/12727 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] tvalentyn commented on pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-23 Thread GitBox
tvalentyn commented on pull request #12727: URL: https://github.com/apache/beam/pull/12727#issuecomment-698036314 I don't have additional comments at this time so I am fine to merge and iterate. This is an automated message

[GitHub] [beam] tvalentyn commented on pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-23 Thread GitBox
tvalentyn commented on pull request #12727: URL: https://github.com/apache/beam/pull/12727#issuecomment-698036221 I looked at codecov errors on the PR, I think the path that codecov reports about are covered by the integration test.

[GitHub] [beam] robinyqiu commented on pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-23 Thread GitBox
robinyqiu commented on pull request #12727: URL: https://github.com/apache/beam/pull/12727#issuecomment-698035756 codecov/patch is failing (what is this?) If this need to be fixed should we move it to 2.26.0? This is an

[GitHub] [beam] TheNeuralBit commented on pull request #12923: Add to_pcollection example to wordcount_dataframe

2020-09-23 Thread GitBox
TheNeuralBit commented on pull request #12923: URL: https://github.com/apache/beam/pull/12923#issuecomment-698032656 R: @robinyqiu This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] rezarokni opened a new pull request #12924: Contextualio rebase

2020-09-23 Thread GitBox
rezarokni opened a new pull request #12924: URL: https://github.com/apache/beam/pull/12924 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution

[GitHub] [beam] TheNeuralBit opened a new pull request #12923: Add to_pcollection example to wordcount_dataframe

2020-09-23 Thread GitBox
TheNeuralBit opened a new pull request #12923: URL: https://github.com/apache/beam/pull/12923 Post-Commit Tests Status (on master branch) Lang | SDK | Dataflow | Flink | Samza | Spark

[GitHub] [beam] aaltay commented on pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-23 Thread GitBox
aaltay commented on pull request #12727: URL: https://github.com/apache/beam/pull/12727#issuecomment-698027243 @tvalentyn - I will defer to you. You provided the last round of feedback. (/cc: @robinyqiu - in case you would like to merge it today for the next release.)

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] y1chi commented on pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-23 Thread GitBox
y1chi commented on pull request #12727: URL: https://github.com/apache/beam/pull/12727#issuecomment-698025433 @aaltay @tvalentyn PTAL and help merge if this LG. This is an automated message from the Apache Git Service. To

[GitHub] [beam] pabloem commented on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
pabloem commented on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-698023942 thanks @robinyqiu @chamikaramj ! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] pabloem merged pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
pabloem merged pull request #12782: URL: https://github.com/apache/beam/pull/12782 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-696899540 # [Codecov](https://codecov.io/gh/apache/beam/pull/12782?src=pr=h1) Report > Merging [#12782](https://codecov.io/gh/apache/beam/pull/12782?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-696899540 # [Codecov](https://codecov.io/gh/apache/beam/pull/12782?src=pr=h1) Report > Merging [#12782](https://codecov.io/gh/apache/beam/pull/12782?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-696899540 # [Codecov](https://codecov.io/gh/apache/beam/pull/12782?src=pr=h1) Report > Merging [#12782](https://codecov.io/gh/apache/beam/pull/12782?src=pr=desc) into

[GitHub] [beam] pabloem commented on pull request #12922: [BEAM-10956] Fixes incorrect API usage in PipelineInstrumentTest causing flakiness

2020-09-23 Thread GitBox
pabloem commented on pull request #12922: URL: https://github.com/apache/beam/pull/12922#issuecomment-698012684 thanks Sam! (LGTM fwiw) This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] pabloem merged pull request #12922: [BEAM-10956] Fixes incorrect API usage in PipelineInstrumentTest causing flakiness

2020-09-23 Thread GitBox
pabloem merged pull request #12922: URL: https://github.com/apache/beam/pull/12922 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] pabloem commented on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
pabloem commented on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-698012362 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] codecov[bot] edited a comment on pull request #12905: [BEAM-8660] Use PortableOptions.artifact_endpoint if provided over value reported from jobserver

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12905: URL: https://github.com/apache/beam/pull/12905#issuecomment-696946990 # [Codecov](https://codecov.io/gh/apache/beam/pull/12905?src=pr=h1) Report > Merging [#12905](https://codecov.io/gh/apache/beam/pull/12905?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-696899540 # [Codecov](https://codecov.io/gh/apache/beam/pull/12782?src=pr=h1) Report > Merging [#12782](https://codecov.io/gh/apache/beam/pull/12782?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-696899540 # [Codecov](https://codecov.io/gh/apache/beam/pull/12782?src=pr=h1) Report > Merging [#12782](https://codecov.io/gh/apache/beam/pull/12782?src=pr=desc) into

[GitHub] [beam] TheNeuralBit merged pull request #12920: [BEAM-9547] Roll forward #12858

2020-09-23 Thread GitBox
TheNeuralBit merged pull request #12920: URL: https://github.com/apache/beam/pull/12920 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-696899540 # [Codecov](https://codecov.io/gh/apache/beam/pull/12782?src=pr=h1) Report > Merging [#12782](https://codecov.io/gh/apache/beam/pull/12782?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12905: [BEAM-8660] Use PortableOptions.artifact_endpoint if provided over value reported from jobserver

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12905: URL: https://github.com/apache/beam/pull/12905#issuecomment-696946990 # [Codecov](https://codecov.io/gh/apache/beam/pull/12905?src=pr=h1) Report > Merging [#12905](https://codecov.io/gh/apache/beam/pull/12905?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12905: [BEAM-8660] Use PortableOptions.artifact_endpoint if provided over value reported from jobserver

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12905: URL: https://github.com/apache/beam/pull/12905#issuecomment-696946990 # [Codecov](https://codecov.io/gh/apache/beam/pull/12905?src=pr=h1) Report > Merging [#12905](https://codecov.io/gh/apache/beam/pull/12905?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-696899540 # [Codecov](https://codecov.io/gh/apache/beam/pull/12782?src=pr=h1) Report > Merging [#12782](https://codecov.io/gh/apache/beam/pull/12782?src=pr=desc) into

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12823: [BEAM-10882] Update Snowflake docs

2020-09-23 Thread GitBox
TheNeuralBit commented on a change in pull request #12823: URL: https://github.com/apache/beam/pull/12823#discussion_r493921269 ## File path: website/www/site/content/en/documentation/io/built-in/snowflake.md ## @@ -362,3 +635,208 @@ static SnowflakeIO.CsvMapper

[GitHub] [beam] codecov[bot] edited a comment on pull request #12905: [BEAM-8660] Use PortableOptions.artifact_endpoint if provided over value reported from jobserver

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12905: URL: https://github.com/apache/beam/pull/12905#issuecomment-696946990 # [Codecov](https://codecov.io/gh/apache/beam/pull/12905?src=pr=h1) Report > Merging [#12905](https://codecov.io/gh/apache/beam/pull/12905?src=pr=desc) into

[GitHub] [beam] pabloem commented on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
pabloem commented on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-697995983 Run Python 3.8 PostCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12823: [BEAM-10882] Update Snowflake docs

2020-09-23 Thread GitBox
TheNeuralBit commented on a change in pull request #12823: URL: https://github.com/apache/beam/pull/12823#discussion_r493908171 ## File path: website/www/site/content/en/documentation/io/built-in/snowflake.md ## @@ -362,3 +635,208 @@ static SnowflakeIO.CsvMapper

[GitHub] [beam] pabloem commented on a change in pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
pabloem commented on a change in pull request #12782: URL: https://github.com/apache/beam/pull/12782#discussion_r493919995 ## File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py ## @@ -269,7 +260,9 @@ def test_biqquery_read_fn_api_fail(self):

[GitHub] [beam] sambvfx commented on pull request #12905: [BEAM-8660] Use PortableOptions.artifact_endpoint if provided over value reported from jobserver

2020-09-23 Thread GitBox
sambvfx commented on pull request #12905: URL: https://github.com/apache/beam/pull/12905#issuecomment-697995387 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] sambvfx commented on a change in pull request #12905: [BEAM-8660] Use PortableOptions.artifact_endpoint if provided over value reported from jobserver

2020-09-23 Thread GitBox
sambvfx commented on a change in pull request #12905: URL: https://github.com/apache/beam/pull/12905#discussion_r493920269 ## File path: sdks/python/apache_beam/runners/portability/portable_runner.py ## @@ -105,9 +106,11 @@ def submit(self, proto_pipeline): Submit and run

[GitHub] [beam] pabloem commented on a change in pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
pabloem commented on a change in pull request #12782: URL: https://github.com/apache/beam/pull/12782#discussion_r493919948 ## File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py ## @@ -323,7 +316,9 @@ def

[GitHub] [beam] pabloem commented on a change in pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
pabloem commented on a change in pull request #12782: URL: https://github.com/apache/beam/pull/12782#discussion_r493917934 ## File path: sdks/python/apache_beam/io/gcp/bigquery_test.py ## @@ -168,7 +168,8 @@ def test_invalid_json_neg_inf(self): @unittest.skipIf(HttpError is

[GitHub] [beam] pabloem commented on a change in pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
pabloem commented on a change in pull request #12782: URL: https://github.com/apache/beam/pull/12782#discussion_r493917759 ## File path: sdks/python/apache_beam/io/gcp/bigquery_tools_test.py ## @@ -451,7 +451,9 @@ def test_read_from_table(self):

[GitHub] [beam] pabloem commented on a change in pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
pabloem commented on a change in pull request #12782: URL: https://github.com/apache/beam/pull/12782#discussion_r493916902 ## File path: sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py ## @@ -91,24 +91,16 @@ def run_bq_pipeline(argv=None):

[GitHub] [beam] robinyqiu commented on pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
robinyqiu commented on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-697991687 Can this PR be merged now? The tests all passed. This is an automated message from the Apache Git Service. To

[GitHub] [beam] kennknowles commented on pull request #12907: [BEAM-10953] Fix logging placeholder mismatches.

2020-09-23 Thread GitBox
kennknowles commented on pull request #12907: URL: https://github.com/apache/beam/pull/12907#issuecomment-697990630 There's also a `MessageFormat` checker which is close but not the same. So it would probably be easy (for the checkerframework team) to add one for SLF4J.

[GitHub] [beam] iemejia commented on pull request #12603: [WIP][BEAM-10670] Make SparkRunner opt-out for using an SDF powered Read transform.

2020-09-23 Thread GitBox
iemejia commented on pull request #12603: URL: https://github.com/apache/beam/pull/12603#issuecomment-697989304 The phenomenon of microbatches producing results early I noticed it too in the past when trying to enable the Read.Unbounded tests. I could not understand why, and I thought it

[GitHub] [beam] kennknowles commented on pull request #12907: [BEAM-10953] Fix logging placeholder mismatches.

2020-09-23 Thread GitBox
kennknowles commented on pull request #12907: URL: https://github.com/apache/beam/pull/12907#issuecomment-697989420 Great question. It turns out that checkerframework [has a checker for printf style strings](https://checkerframework.org/manual/#formatter-checker) but I don't know about

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12823: [BEAM-10882] Update Snowflake docs

2020-09-23 Thread GitBox
TheNeuralBit commented on a change in pull request #12823: URL: https://github.com/apache/beam/pull/12823#discussion_r493907866 ## File path: website/www/site/content/en/documentation/io/built-in/snowflake.md ## @@ -362,3 +635,208 @@ static SnowflakeIO.CsvMapper

[GitHub] [beam] pabloem merged pull request #12912: [BEAM-10938] Adds support for writing a footer to Python WriteToText

2020-09-23 Thread GitBox
pabloem merged pull request #12912: URL: https://github.com/apache/beam/pull/12912 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12912: [BEAM-10938] Adds support for writing a footer to Python WriteToText

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12912: URL: https://github.com/apache/beam/pull/12912#issuecomment-697965672 # [Codecov](https://codecov.io/gh/apache/beam/pull/12912?src=pr=h1) Report > Merging [#12912](https://codecov.io/gh/apache/beam/pull/12912?src=pr=desc) into

[GitHub] [beam] tysonjh commented on a change in pull request #12915: [BEAM-7386] Introduce temporal inner join.

2020-09-23 Thread GitBox
tysonjh commented on a change in pull request #12915: URL: https://github.com/apache/beam/pull/12915#discussion_r493905016 ## File path: sdks/java/extensions/join-library/src/main/java/org/apache/beam/sdk/extensions/joinlibrary/Join.java ## @@ -350,6 +368,276 @@ public void

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] tysonjh commented on pull request #12915: [BEAM-7386] Introduce temporal inner join.

2020-09-23 Thread GitBox
tysonjh commented on pull request #12915: URL: https://github.com/apache/beam/pull/12915#issuecomment-697981031 > Out of curiosity, why are you adding this here instead of the schema join library (which SQL uses)? I wasn't aware of the other join library. I saw the join extension

[GitHub] [beam] codecov[bot] edited a comment on pull request #12912: [BEAM-10938] Adds support for writing a footer to Python WriteToText

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12912: URL: https://github.com/apache/beam/pull/12912#issuecomment-697965672 # [Codecov](https://codecov.io/gh/apache/beam/pull/12912?src=pr=h1) Report > Merging [#12912](https://codecov.io/gh/apache/beam/pull/12912?src=pr=desc) into

[GitHub] [beam] pabloem commented on pull request #12912: [BEAM-10938] Adds support for writing a footer to Python WriteToText

2020-09-23 Thread GitBox
pabloem commented on pull request #12912: URL: https://github.com/apache/beam/pull/12912#issuecomment-697975445 Run Portable_Python PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] chamikaramj merged pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
chamikaramj merged pull request #12914: URL: https://github.com/apache/beam/pull/12914 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] codecov[bot] edited a comment on pull request #12912: [BEAM-10938] Adds support for writing a footer to Python WriteToText

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12912: URL: https://github.com/apache/beam/pull/12912#issuecomment-697965672 # [Codecov](https://codecov.io/gh/apache/beam/pull/12912?src=pr=h1) Report > Merging [#12912](https://codecov.io/gh/apache/beam/pull/12912?src=pr=desc) into

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12838: [BEAM-10892] Add Proto support to Kafka Table Provider

2020-09-23 Thread GitBox
TheNeuralBit commented on a change in pull request #12838: URL: https://github.com/apache/beam/pull/12838#discussion_r493890826 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/kafka/BeamKafkaProtoTable.java ## @@ -0,0

[GitHub] [beam] tysonjh commented on a change in pull request #12904: Add link issue for OrderedListState.

2020-09-23 Thread GitBox
tysonjh commented on a change in pull request #12904: URL: https://github.com/apache/beam/pull/12904#discussion_r493890474 ## File path: runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java ## @@ -2044,6 +2044,7 @@ static void

[GitHub] [beam] codecov[bot] edited a comment on pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12914: URL: https://github.com/apache/beam/pull/12914#issuecomment-697498651 # [Codecov](https://codecov.io/gh/apache/beam/pull/12914?src=pr=h1) Report > Merging [#12914](https://codecov.io/gh/apache/beam/pull/12914?src=pr=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12912: [BEAM-10938] Adds support for writing a footer to Python WriteToText

2020-09-23 Thread GitBox
codecov[bot] commented on pull request #12912: URL: https://github.com/apache/beam/pull/12912#issuecomment-697965672 # [Codecov](https://codecov.io/gh/apache/beam/pull/12912?src=pr=h1) Report > Merging [#12912](https://codecov.io/gh/apache/beam/pull/12912?src=pr=desc) into

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12827: [BEAM-10885] Add Avro support to Kafka table provider

2020-09-23 Thread GitBox
TheNeuralBit commented on a change in pull request #12827: URL: https://github.com/apache/beam/pull/12827#discussion_r493858640 ## File path: sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider/kafka/BeamKafkaTableTest.java ## @@ -41,27

[GitHub] [beam] codecov[bot] edited a comment on pull request #12922: [BEAM-10956] Fixes incorrect API usage in PipelineInstrumentTest causing flakiness

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12922: URL: https://github.com/apache/beam/pull/12922#issuecomment-697941809 # [Codecov](https://codecov.io/gh/apache/beam/pull/12922?src=pr=h1) Report > Merging [#12922](https://codecov.io/gh/apache/beam/pull/12922?src=pr=desc) into

[GitHub] [beam] chamikaramj commented on pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
chamikaramj commented on pull request #12914: URL: https://github.com/apache/beam/pull/12914#issuecomment-697952427 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] tvalentyn merged pull request #12524: [BEAM-10586] Remove Python 2.7 and Python 3.5 support in Dataflow runner.

2020-09-23 Thread GitBox
tvalentyn merged pull request #12524: URL: https://github.com/apache/beam/pull/12524 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] codecov[bot] edited a comment on pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12914: URL: https://github.com/apache/beam/pull/12914#issuecomment-697498651 # [Codecov](https://codecov.io/gh/apache/beam/pull/12914?src=pr=h1) Report > Merging [#12914](https://codecov.io/gh/apache/beam/pull/12914?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12920: [BEAM-9547] Roll forward #12858

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12920: URL: https://github.com/apache/beam/pull/12920#issuecomment-697872427 # [Codecov](https://codecov.io/gh/apache/beam/pull/12920?src=pr=h1) Report > Merging [#12920](https://codecov.io/gh/apache/beam/pull/12920?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12914: URL: https://github.com/apache/beam/pull/12914#issuecomment-697498651 # [Codecov](https://codecov.io/gh/apache/beam/pull/12914?src=pr=h1) Report > Merging [#12914](https://codecov.io/gh/apache/beam/pull/12914?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12922: [BEAM-10956] Fixes incorrect API usage in PipelineInstrumentTest causing flakiness

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12922: URL: https://github.com/apache/beam/pull/12922#issuecomment-697941809 # [Codecov](https://codecov.io/gh/apache/beam/pull/12922?src=pr=h1) Report > Merging [#12922](https://codecov.io/gh/apache/beam/pull/12922?src=pr=desc) into

[GitHub] [beam] chamikaramj commented on a change in pull request #12782: [BEAM-10950] Overriding Dataflow Native BQSource.

2020-09-23 Thread GitBox
chamikaramj commented on a change in pull request #12782: URL: https://github.com/apache/beam/pull/12782#discussion_r493864558 ## File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py ## @@ -269,7 +260,9 @@ def test_biqquery_read_fn_api_fail(self):

[GitHub] [beam] codecov[bot] edited a comment on pull request #12920: [BEAM-9547] Roll forward #12858

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12920: URL: https://github.com/apache/beam/pull/12920#issuecomment-697872427 # [Codecov](https://codecov.io/gh/apache/beam/pull/12920?src=pr=h1) Report > Merging [#12920](https://codecov.io/gh/apache/beam/pull/12920?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12524: [BEAM-10586] Remove Python 2.7 and Python 3.5 support in Dataflow runner.

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12524: URL: https://github.com/apache/beam/pull/12524#issuecomment-697896263 # [Codecov](https://codecov.io/gh/apache/beam/pull/12524?src=pr=h1) Report > Merging [#12524](https://codecov.io/gh/apache/beam/pull/12524?src=pr=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12922: [BEAM-10956] Fixes incorrect API usage in PipelineInstrumentTest causing flakiness

2020-09-23 Thread GitBox
codecov[bot] commented on pull request #12922: URL: https://github.com/apache/beam/pull/12922#issuecomment-697941809 # [Codecov](https://codecov.io/gh/apache/beam/pull/12922?src=pr=h1) Report > Merging [#12922](https://codecov.io/gh/apache/beam/pull/12922?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12914: URL: https://github.com/apache/beam/pull/12914#issuecomment-697498651 # [Codecov](https://codecov.io/gh/apache/beam/pull/12914?src=pr=h1) Report > Merging [#12914](https://codecov.io/gh/apache/beam/pull/12914?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12914: URL: https://github.com/apache/beam/pull/12914#issuecomment-697498651 # [Codecov](https://codecov.io/gh/apache/beam/pull/12914?src=pr=h1) Report > Merging [#12914](https://codecov.io/gh/apache/beam/pull/12914?src=pr=desc) into

[GitHub] [beam] rohdesamuel commented on pull request #12922: [BEAM-10956] Fixes incorrect API usage in PipelineInstrumentTest causing flakiness

2020-09-23 Thread GitBox
rohdesamuel commented on pull request #12922: URL: https://github.com/apache/beam/pull/12922#issuecomment-697938137 Verified fix with `for i in {1..100}; do python pipeline_instrument_test.py & done` This is an automated

[GitHub] [beam] codecov[bot] edited a comment on pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12914: URL: https://github.com/apache/beam/pull/12914#issuecomment-697498651 # [Codecov](https://codecov.io/gh/apache/beam/pull/12914?src=pr=h1) Report > Merging [#12914](https://codecov.io/gh/apache/beam/pull/12914?src=pr=desc) into

[GitHub] [beam] rohdesamuel opened a new pull request #12922: Fix BEAM-10956

2020-09-23 Thread GitBox
rohdesamuel opened a new pull request #12922: URL: https://github.com/apache/beam/pull/12922 Change-Id: Ied7a07ce16ebb1e155692666335a6dae498a01ea Test was using the mark_pcollection_computed method incorrectly. Thank you for your contribution!

[GitHub] [beam] chamikaramj commented on a change in pull request #12888: [BEAM-10861]Moves PubSub Runner API encoding to Read/Write transforms

2020-09-23 Thread GitBox
chamikaramj commented on a change in pull request #12888: URL: https://github.com/apache/beam/pull/12888#discussion_r493850984 ## File path: sdks/python/apache_beam/io/iobase.py ## @@ -992,6 +1012,43 @@ def expand(self, pcoll): 'A sink must inherit iobase.Sink,

[GitHub] [beam] codecov[bot] edited a comment on pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
codecov[bot] edited a comment on pull request #12914: URL: https://github.com/apache/beam/pull/12914#issuecomment-697498651 # [Codecov](https://codecov.io/gh/apache/beam/pull/12914?src=pr=h1) Report > Merging [#12914](https://codecov.io/gh/apache/beam/pull/12914?src=pr=desc) into

[GitHub] [beam] chamikaramj commented on pull request #12914: [BEAM-10861] Revert "[BEAM-10861] Adds URNs and payloads to PubSub transforms to allow runner-native overrides"

2020-09-23 Thread GitBox
chamikaramj commented on pull request #12914: URL: https://github.com/apache/beam/pull/12914#issuecomment-697932085 R: @boyuanzz This is an automated message from the Apache Git Service. To respond to the message, please

  1   2   3   >