[GitHub] [beam] codecov[bot] edited a comment on pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12505: URL: https://github.com/apache/beam/pull/12505#issuecomment-684502100 # [Codecov](https://codecov.io/gh/apache/beam/pull/12505?src=pr=h1) Report > Merging [#12505](https://codecov.io/gh/apache/beam/pull/12505?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12505: URL: https://github.com/apache/beam/pull/12505#issuecomment-684502100 # [Codecov](https://codecov.io/gh/apache/beam/pull/12505?src=pr=h1) Report > Merging [#12505](https://codecov.io/gh/apache/beam/pull/12505?src=pr=desc) into

[GitHub] [beam] aromanenko-dev opened a new pull request #12862: [BEAM-10816] Make KinesisClientThrottledException public

2020-09-17 Thread GitBox
aromanenko-dev opened a new pull request #12862: URL: https://github.com/apache/beam/pull/12862 `KinesisClientThrottledException` is a type of argument used in `RateLimitPolicy.onThrottle()`, which should be implemented by user in case of using custom `RateLimitPolicy`. So, it should be

[GitHub] [beam] piotr-szuberski commented on pull request #12847: [BEAM-7523] Fix starting Kafka container twice in KafkaCSVTableIT

2020-09-17 Thread GitBox
piotr-szuberski commented on pull request #12847: URL: https://github.com/apache/beam/pull/12847#issuecomment-694099897 > Sorry to hear about the food poisoning, hope you're feeling better now :) I was up and running the next day, thank you :)

[GitHub] [beam] kmjung commented on a change in pull request #12767: [BEAM-10916] Remove experimental annotations for BQ storage API source

2020-09-17 Thread GitBox
kmjung commented on a change in pull request #12767: URL: https://github.com/apache/beam/pull/12767#discussion_r489989080 ## File path: website/www/site/content/en/documentation/io/built-in/google-bigquery.md ## @@ -322,21 +322,17 @@ in the following example: ### Using the

[GitHub] [beam] codecov[bot] edited a comment on pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12505: URL: https://github.com/apache/beam/pull/12505#issuecomment-684502100 # [Codecov](https://codecov.io/gh/apache/beam/pull/12505?src=pr=h1) Report > Merging [#12505](https://codecov.io/gh/apache/beam/pull/12505?src=pr=desc) into

[GitHub] [beam] piotr-szuberski commented on pull request #12809: [BEAM-10870] Add raw private key param to snowflake cross-language python wrapper

2020-09-17 Thread GitBox
piotr-szuberski commented on pull request #12809: URL: https://github.com/apache/beam/pull/12809#issuecomment-694101221 @purbanow Could you review this PR? This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] aromanenko-dev commented on pull request #12862: [BEAM-10816] Make KinesisClientThrottledException public

2020-09-17 Thread GitBox
aromanenko-dev commented on pull request #12862: URL: https://github.com/apache/beam/pull/12862#issuecomment-694101244 I'm wondering if `Spotless` or other checks that we have should detect such things. This is an automated

[GitHub] [beam] codecov[bot] commented on pull request #12866: [BEAM-10921] Fix flaky unit tests on Windows

2020-09-17 Thread GitBox
codecov[bot] commented on pull request #12866: URL: https://github.com/apache/beam/pull/12866#issuecomment-694408723 # [Codecov](https://codecov.io/gh/apache/beam/pull/12866?src=pr=h1) Report > Merging [#12866](https://codecov.io/gh/apache/beam/pull/12866?src=pr=desc) into

[GitHub] [beam] TheNeuralBit commented on pull request #12856: [BEAM-9547] Add indexing tests

2020-09-17 Thread GitBox
TheNeuralBit commented on pull request #12856: URL: https://github.com/apache/beam/pull/12856#issuecomment-694447794 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] aaltay commented on pull request #12735: [BEAM-9543] Add blog post for MATCH_RECOGNIZE

2020-09-17 Thread GitBox
aaltay commented on pull request #12735: URL: https://github.com/apache/beam/pull/12735#issuecomment-694467358 @amaliujia - what is the next step on this PR? This is an automated message from the Apache Git Service. To

[GitHub] [beam] aaltay commented on pull request #12649: [BEAM-10778] Integration tests for Azure filesystem

2020-09-17 Thread GitBox
aaltay commented on pull request #12649: URL: https://github.com/apache/beam/pull/12649#issuecomment-694467154 @pabloem - what is the next step on this PR? This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] aaltay commented on pull request #12647: [BEAM-10378] Deserializing Azure Credentials

2020-09-17 Thread GitBox
aaltay commented on pull request #12647: URL: https://github.com/apache/beam/pull/12647#issuecomment-694467117 @pabloem - what is the next step on this PR? This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] codecov[bot] edited a comment on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692960218 # [Codecov](https://codecov.io/gh/apache/beam/pull/12799?src=pr=h1) Report > Merging [#12799](https://codecov.io/gh/apache/beam/pull/12799?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12727: URL: https://github.com/apache/beam/pull/12727#issuecomment-683229966 # [Codecov](https://codecov.io/gh/apache/beam/pull/12727?src=pr=h1) Report > Merging [#12727](https://codecov.io/gh/apache/beam/pull/12727?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12866: [BEAM-10921] Fix flaky unit tests on Windows

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12866: URL: https://github.com/apache/beam/pull/12866#issuecomment-694408723 # [Codecov](https://codecov.io/gh/apache/beam/pull/12866?src=pr=h1) Report > Merging [#12866](https://codecov.io/gh/apache/beam/pull/12866?src=pr=desc) into

[GitHub] [beam] TheNeuralBit merged pull request #12866: [BEAM-10921] Fix flaky unit tests on Windows

2020-09-17 Thread GitBox
TheNeuralBit merged pull request #12866: URL: https://github.com/apache/beam/pull/12866 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [beam] aaltay commented on pull request #12595: [BEAM-9198] BeamSQL aggregation analytics functionality : Add support for Analytic Functions in ZetaSQL

2020-09-17 Thread GitBox
aaltay commented on pull request #12595: URL: https://github.com/apache/beam/pull/12595#issuecomment-694467327 @amaliujia - what is the next step on this PR? This is an automated message from the Apache Git Service. To

[GitHub] [beam] aaltay commented on pull request #12790: [BEAM-10862] Handle empty tfrecord files within a glob

2020-09-17 Thread GitBox
aaltay commented on pull request #12790: URL: https://github.com/apache/beam/pull/12790#issuecomment-694467525 @chamikaramj - what is the next step on this PR? This is an automated message from the Apache Git Service. To

[GitHub] [beam] boyuanzz commented on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
boyuanzz commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-694470291 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] boyuanzz commented on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
boyuanzz commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-694469970 retest all please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] amaliujia commented on pull request #12595: [BEAM-9198] BeamSQL aggregation analytics functionality : Add support for Analytic Functions in ZetaSQL

2020-09-17 Thread GitBox
amaliujia commented on pull request #12595: URL: https://github.com/apache/beam/pull/12595#issuecomment-694485388 @aaltay this change will break internal test suite. Need to take a closer to know why the suite is broken and how to fix.

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12727: URL: https://github.com/apache/beam/pull/12727#issuecomment-683229966 # [Codecov](https://codecov.io/gh/apache/beam/pull/12727?src=pr=h1) Report > Merging [#12727](https://codecov.io/gh/apache/beam/pull/12727?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12727: URL: https://github.com/apache/beam/pull/12727#issuecomment-683229966 # [Codecov](https://codecov.io/gh/apache/beam/pull/12727?src=pr=h1) Report > Merging [#12727](https://codecov.io/gh/apache/beam/pull/12727?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692960218 # [Codecov](https://codecov.io/gh/apache/beam/pull/12799?src=pr=h1) Report > Merging [#12799](https://codecov.io/gh/apache/beam/pull/12799?src=pr=desc) into

[GitHub] [beam] TheNeuralBit commented on pull request #12866: [BEAM-10921] Fix flaky unit tests on Windows

2020-09-17 Thread GitBox
TheNeuralBit commented on pull request #12866: URL: https://github.com/apache/beam/pull/12866#issuecomment-694403807 Python macos failures look like known flakes This is an automated message from the Apache Git Service. To

[GitHub] [beam] codecov[bot] edited a comment on pull request #12866: [BEAM-10921] Fix flaky unit tests on Windows

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12866: URL: https://github.com/apache/beam/pull/12866#issuecomment-694408723 # [Codecov](https://codecov.io/gh/apache/beam/pull/12866?src=pr=h1) Report > Merging [#12866](https://codecov.io/gh/apache/beam/pull/12866?src=pr=desc) into

[GitHub] [beam] NikeNano closed pull request #12842: Update quickstart-java.md to used -P correctly in the documentations.

2020-09-17 Thread GitBox
NikeNano closed pull request #12842: URL: https://github.com/apache/beam/pull/12842 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] aaltay commented on pull request #11582: [BEAM-9650] Add ReadAllFromBigQuery PTransform

2020-09-17 Thread GitBox
aaltay commented on pull request #11582: URL: https://github.com/apache/beam/pull/11582#issuecomment-694465408 Is this stale? Should we keep it open? This is an automated message from the Apache Git Service. To respond to

[GitHub] [beam] aaltay commented on pull request #12678: [BEAM-10703] Add a step property for shardable states during Dataflow graph translation (Java)

2020-09-17 Thread GitBox
aaltay commented on pull request #12678: URL: https://github.com/apache/beam/pull/12678#issuecomment-694465118 Is this ready to be merged? This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] codecov[bot] edited a comment on pull request #12856: [BEAM-9547] Add indexing tests

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12856: URL: https://github.com/apache/beam/pull/12856#issuecomment-694460157 # [Codecov](https://codecov.io/gh/apache/beam/pull/12856?src=pr=h1) Report > Merging [#12856](https://codecov.io/gh/apache/beam/pull/12856?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] rohdesamuel commented on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-17 Thread GitBox
rohdesamuel commented on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-694484142 > > The failing test seems to be a related test in Windows: https://github.com/apache/beam/pull/12799/checks?check_run_id=1119719710 > > Can you check whether it's related

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] TheNeuralBit closed pull request #12865: [BEAM-10921] Revert "Merge pull request #12704 from [BEAM-10603] Implement the new…

2020-09-17 Thread GitBox
TheNeuralBit closed pull request #12865: URL: https://github.com/apache/beam/pull/12865 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12727: URL: https://github.com/apache/beam/pull/12727#issuecomment-683229966 # [Codecov](https://codecov.io/gh/apache/beam/pull/12727?src=pr=h1) Report > Merging [#12727](https://codecov.io/gh/apache/beam/pull/12727?src=pr=desc) into

[GitHub] [beam] rohdesamuel commented on pull request #12866: [BEAM-10921] Fix flaky unit tests on Windows

2020-09-17 Thread GitBox
rohdesamuel commented on pull request #12866: URL: https://github.com/apache/beam/pull/12866#issuecomment-694404100 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] TheNeuralBit commented on pull request #12865: [BEAM-10921] Revert "Merge pull request #12704 from [BEAM-10603] Implement the new…

2020-09-17 Thread GitBox
TheNeuralBit commented on pull request #12865: URL: https://github.com/apache/beam/pull/12865#issuecomment-694404869 Closing in favor of #12866 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] boyuanzz commented on a change in pull request #12836: [BEAM-10887] Expose clear() to FnApiTimer.

2020-09-17 Thread GitBox
boyuanzz commented on a change in pull request #12836: URL: https://github.com/apache/beam/pull/12836#discussion_r490541733 ## File path: runners/direct-java/src/main/java/org/apache/beam/runners/direct/WatermarkManager.java ## @@ -1624,7 +1630,15 @@ public TimerUpdateBuilder

[GitHub] [beam] amaliujia commented on a change in pull request #12735: [BEAM-9543] Add blog post for MATCH_RECOGNIZE

2020-09-17 Thread GitBox
amaliujia commented on a change in pull request #12735: URL: https://github.com/apache/beam/pull/12735#discussion_r490542490 ## File path: website/www/site/content/en/blog/pattern-match-beam-sql.md ## @@ -0,0 +1,167 @@ +--- +title: "Pattern Matching with Beam SQL" +date:

[GitHub] [beam] rohdesamuel opened a new pull request #12866: [BEAM-10921] Fix flaky unit tests on Windows

2020-09-17 Thread GitBox
rohdesamuel opened a new pull request #12866: URL: https://github.com/apache/beam/pull/12866 Change-Id: I6a4bbd8c2f56d10e40a9f0dcb32b8bfa33549214 The tests are run in parallel causing the same environment to be cleaned up in multiple threads. This keeps the same environment.

[GitHub] [beam] codecov[bot] edited a comment on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692960218 # [Codecov](https://codecov.io/gh/apache/beam/pull/12799?src=pr=h1) Report > Merging [#12799](https://codecov.io/gh/apache/beam/pull/12799?src=pr=desc) into

[GitHub] [beam] robertwb commented on a change in pull request #12858: [BEAM-9547][WIP] Raise NotImplementedError and WontImplementError throughout DeferredDataframe

2020-09-17 Thread GitBox
robertwb commented on a change in pull request #12858: URL: https://github.com/apache/beam/pull/12858#discussion_r490028277 ## File path: sdks/python/apache_beam/dataframe/frames.py ## @@ -763,8 +788,9 @@ def sort_values(self, axis, **kwargs): transform =

[GitHub] [beam] codecov[bot] commented on pull request #12856: [BEAM-9547] Add indexing tests

2020-09-17 Thread GitBox
codecov[bot] commented on pull request #12856: URL: https://github.com/apache/beam/pull/12856#issuecomment-694460157 # [Codecov](https://codecov.io/gh/apache/beam/pull/12856?src=pr=h1) Report > Merging [#12856](https://codecov.io/gh/apache/beam/pull/12856?src=pr=desc) into

[GitHub] [beam] boyuanzz commented on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
boyuanzz commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-694470173 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] amaliujia merged pull request #12735: [BEAM-9543] Add blog post for MATCH_RECOGNIZE

2020-09-17 Thread GitBox
amaliujia merged pull request #12735: URL: https://github.com/apache/beam/pull/12735 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] palmere-google opened a new pull request #12867: Pub/Sub Lite getSplitBacklog implementation

2020-09-17 Thread GitBox
palmere-google opened a new pull request #12867: URL: https://github.com/apache/beam/pull/12867 Add an implementation of getSplitBacklog to the Pub/Sub Lite Unbounded Reader. This implementation depends on the pub/sub lite TopicStats api. Thank you

[GitHub] [beam] aaltay commented on a change in pull request #12745: [DO NOT MERGE] Add a blog post for Apache Beam 2.24.0.

2020-09-17 Thread GitBox
aaltay commented on a change in pull request #12745: URL: https://github.com/apache/beam/pull/12745#discussion_r490562478 ## File path: website/www/site/content/en/blog/beam-2.24.0.md ## @@ -0,0 +1,72 @@ +--- +title: "Apache Beam 2.24.0" +date: 2020-07-29 00:00:01 -0800 +#

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] KevinGG commented on a change in pull request #12704: [BEAM-10603] Implement the new Large Source Recording API.

2020-09-17 Thread GitBox
KevinGG commented on a change in pull request #12704: URL: https://github.com/apache/beam/pull/12704#discussion_r490593309 ## File path: sdks/python/apache_beam/runners/interactive/display/pcoll_visualization.py ## @@ -407,13 +392,7 @@ def _display_dataframe(self, data,

[GitHub] [beam] ibzib commented on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-17 Thread GitBox
ibzib commented on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-694540273 > I'm a little unsure about the overhead this adds in terms of the number of options. Would it make sense to instead use the following format? For example: > > ``` >

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr=desc) into

[GitHub] [beam] boyuanzz merged pull request #12678: [BEAM-10703] Add a step property for shardable states during Dataflow graph translation (Java)

2020-09-17 Thread GitBox
boyuanzz merged pull request #12678: URL: https://github.com/apache/beam/pull/12678 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] robertwb merged pull request #12491: Avoid re-encoding row types.

2020-09-17 Thread GitBox
robertwb merged pull request #12491: URL: https://github.com/apache/beam/pull/12491 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692960218 # [Codecov](https://codecov.io/gh/apache/beam/pull/12799?src=pr=h1) Report > Merging [#12799](https://codecov.io/gh/apache/beam/pull/12799?src=pr=desc) into

[GitHub] [beam] boyuanzz commented on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
boyuanzz commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-694547392 Run PythonDocker PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] boyuanzz commented on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
boyuanzz commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-694547267 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12841: [BEAM-10894] Basic CSV reading and writing.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12841: URL: https://github.com/apache/beam/pull/12841#issuecomment-692395796 # [Codecov](https://codecov.io/gh/apache/beam/pull/12841?src=pr=h1) Report > Merging [#12841](https://codecov.io/gh/apache/beam/pull/12841?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] robertwb commented on pull request #12844: [BEAM-10894] Support for more pandas formats.

2020-09-17 Thread GitBox
robertwb commented on pull request #12844: URL: https://github.com/apache/beam/pull/12844#issuecomment-694559264 Rebased on https://github.com/apache/beam/pull/12841 This is an automated message from the Apache Git Service.

[GitHub] [beam] codecov[bot] edited a comment on pull request #12841: [BEAM-10894] Basic CSV reading and writing.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12841: URL: https://github.com/apache/beam/pull/12841#issuecomment-692395796 # [Codecov](https://codecov.io/gh/apache/beam/pull/12841?src=pr=h1) Report > Merging [#12841](https://codecov.io/gh/apache/beam/pull/12841?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12841: [BEAM-10894] Basic CSV reading and writing.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12841: URL: https://github.com/apache/beam/pull/12841#issuecomment-692395796 # [Codecov](https://codecov.io/gh/apache/beam/pull/12841?src=pr=h1) Report > Merging [#12841](https://codecov.io/gh/apache/beam/pull/12841?src=pr=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12844: [BEAM-10894] Support for more pandas formats.

2020-09-17 Thread GitBox
codecov[bot] commented on pull request #12844: URL: https://github.com/apache/beam/pull/12844#issuecomment-694568117 # [Codecov](https://codecov.io/gh/apache/beam/pull/12844?src=pr=h1) Report > Merging [#12844](https://codecov.io/gh/apache/beam/pull/12844?src=pr=desc) into

[GitHub] [beam] youngoli commented on a change in pull request #12743: [DO NOT MERGE] Update Beam website to release 2.24.0.

2020-09-17 Thread GitBox
youngoli commented on a change in pull request #12743: URL: https://github.com/apache/beam/pull/12743#discussion_r490569708 ## File path: website/www/site/content/en/get-started/downloads.md ## @@ -87,16 +87,24 @@ versions denoted `0.x.y`. ## Releases +### 2.24.0

[GitHub] [beam] udim commented on a change in pull request #12745: Add a blog post for Apache Beam 2.24.0.

2020-09-17 Thread GitBox
udim commented on a change in pull request #12745: URL: https://github.com/apache/beam/pull/12745#discussion_r490572359 ## File path: website/www/site/content/en/blog/beam-2.24.0.md ## @@ -0,0 +1,77 @@ +--- +title: "Apache Beam 2.24.0" +date: 2020-09-17 00:00:01 -0800

[GitHub] [beam] udim commented on a change in pull request #12745: Add a blog post for Apache Beam 2.24.0.

2020-09-17 Thread GitBox
udim commented on a change in pull request #12745: URL: https://github.com/apache/beam/pull/12745#discussion_r490571910 ## File path: website/www/site/content/en/blog/beam-2.24.0.md ## @@ -0,0 +1,72 @@ +--- +title: "Apache Beam 2.24.0" +date: 2020-07-29 00:00:01 -0800 +#

[GitHub] [beam] udim commented on a change in pull request #12745: Add a blog post for Apache Beam 2.24.0.

2020-09-17 Thread GitBox
udim commented on a change in pull request #12745: URL: https://github.com/apache/beam/pull/12745#discussion_r490571910 ## File path: website/www/site/content/en/blog/beam-2.24.0.md ## @@ -0,0 +1,72 @@ +--- +title: "Apache Beam 2.24.0" +date: 2020-07-29 00:00:01 -0800 +#

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] robertwb merged pull request #12832: [BEAM-10906] Add basic ToRows transform.

2020-09-17 Thread GitBox
robertwb merged pull request #12832: URL: https://github.com/apache/beam/pull/12832 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692960218 # [Codecov](https://codecov.io/gh/apache/beam/pull/12799?src=pr=h1) Report > Merging [#12799](https://codecov.io/gh/apache/beam/pull/12799?src=pr=desc) into

[GitHub] [beam] youngoli commented on pull request #12743: Update Beam website to release 2.24.0.

2020-09-17 Thread GitBox
youngoli commented on pull request #12743: URL: https://github.com/apache/beam/pull/12743#issuecomment-694507294 This should be safe to merge, but I'm just waiting a bit until the dist.apache link is good This is an

[GitHub] [beam] robertwb commented on pull request #12491: Avoid re-encoding row types.

2020-09-17 Thread GitBox
robertwb commented on pull request #12491: URL: https://github.com/apache/beam/pull/12491#issuecomment-694536569 Run PythonDocker PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12809: [BEAM-10870] Add raw private key param to snowflake cross-language python wrapper

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12809: URL: https://github.com/apache/beam/pull/12809#issuecomment-690248992 # [Codecov](https://codecov.io/gh/apache/beam/pull/12809?src=pr=h1) Report > Merging [#12809](https://codecov.io/gh/apache/beam/pull/12809?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12809: [BEAM-10870] Add raw private key param to snowflake cross-language python wrapper

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12809: URL: https://github.com/apache/beam/pull/12809#issuecomment-690248992 # [Codecov](https://codecov.io/gh/apache/beam/pull/12809?src=pr=h1) Report > Merging [#12809](https://codecov.io/gh/apache/beam/pull/12809?src=pr=desc) into

[GitHub] [beam] kamilwu commented on pull request #12863: [BEAM-10643] Major rearrangement of Nexmark dashboards

2020-09-17 Thread GitBox
kamilwu commented on pull request #12863: URL: https://github.com/apache/beam/pull/12863#issuecomment-694171251 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] aromanenko-dev edited a comment on pull request #12862: [BEAM-10816] Make KinesisClientThrottledException public

2020-09-17 Thread GitBox
aromanenko-dev edited a comment on pull request #12862: URL: https://github.com/apache/beam/pull/12862#issuecomment-694101244 I'm wondering if `Spotless` or other checks, that we have, should detect such things. This is an

[GitHub] [beam] kamilwu opened a new pull request #12863: [BEAM-10643] Major rearrangement of Nexmark dashboards

2020-09-17 Thread GitBox
kamilwu opened a new pull request #12863: URL: https://github.com/apache/beam/pull/12863 This PR removes runner vs. runner comparison, which wasn't particularly useful. Instead, dashboards now compare different implementations for the same query (Standard, SQL or ZetaSQL). For more

[GitHub] [beam] purbanow commented on pull request #12809: [BEAM-10870] Add raw private key param to snowflake cross-language python wrapper

2020-09-17 Thread GitBox
purbanow commented on pull request #12809: URL: https://github.com/apache/beam/pull/12809#issuecomment-694104966 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12809: [BEAM-10870] Add raw private key param to snowflake cross-language python wrapper

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12809: URL: https://github.com/apache/beam/pull/12809#issuecomment-690248992 # [Codecov](https://codecov.io/gh/apache/beam/pull/12809?src=pr=h1) Report > Merging [#12809](https://codecov.io/gh/apache/beam/pull/12809?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12809: [BEAM-10870] Add raw private key param to snowflake cross-language python wrapper

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12809: URL: https://github.com/apache/beam/pull/12809#issuecomment-690248992 # [Codecov](https://codecov.io/gh/apache/beam/pull/12809?src=pr=h1) Report > Merging [#12809](https://codecov.io/gh/apache/beam/pull/12809?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12809: [BEAM-10870] Add raw private key param to snowflake cross-language python wrapper

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12809: URL: https://github.com/apache/beam/pull/12809#issuecomment-690248992 # [Codecov](https://codecov.io/gh/apache/beam/pull/12809?src=pr=h1) Report > Merging [#12809](https://codecov.io/gh/apache/beam/pull/12809?src=pr=desc) into

[GitHub] [beam] mwalenia merged pull request #12809: [BEAM-10870] Add raw private key param to snowflake cross-language python wrapper

2020-09-17 Thread GitBox
mwalenia merged pull request #12809: URL: https://github.com/apache/beam/pull/12809 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12809: [BEAM-10870] Add raw private key param to snowflake cross-language python wrapper

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12809: URL: https://github.com/apache/beam/pull/12809#issuecomment-690248992 # [Codecov](https://codecov.io/gh/apache/beam/pull/12809?src=pr=h1) Report > Merging [#12809](https://codecov.io/gh/apache/beam/pull/12809?src=pr=desc) into

[GitHub] [beam] jayendra13 commented on a change in pull request #12596: [BEAM-10498] [WIP] Eliminate nullability errors from :sdks:java:extensions:sql:zetasql

2020-09-17 Thread GitBox
jayendra13 commented on a change in pull request #12596: URL: https://github.com/apache/beam/pull/12596#discussion_r490648800 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/SqlAnalyzer.java ## @@ -218,6 +218,8 @@

[GitHub] [beam] codecov[bot] edited a comment on pull request #12844: [BEAM-10894] Support for more pandas formats.

2020-09-17 Thread GitBox
codecov[bot] edited a comment on pull request #12844: URL: https://github.com/apache/beam/pull/12844#issuecomment-694568117 # [Codecov](https://codecov.io/gh/apache/beam/pull/12844?src=pr=h1) Report > Merging [#12844](https://codecov.io/gh/apache/beam/pull/12844?src=pr=desc) into

[GitHub] [beam] robertwb commented on a change in pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

2020-09-17 Thread GitBox
robertwb commented on a change in pull request #12727: URL: https://github.com/apache/beam/pull/12727#discussion_r490632829 ## File path: sdks/python/apache_beam/options/pipeline_options.py ## @@ -994,6 +994,23 @@ def _add_argparse_args(cls, parser): 'staged in

[GitHub] [beam] udim commented on pull request #12805: [BEAM-10867] Add file generation to GcsPath

2020-09-17 Thread GitBox
udim commented on pull request #12805: URL: https://github.com/apache/beam/pull/12805#issuecomment-694605629 > Based on an offline chat, seems like we are adding support for GCS versioning here. It'll be good if we can can have a short doc and/or an email to to dev list to make sure we

  1   2   >