[GitHub] [beam] pawelpasterz commented on a change in pull request #10882: Implement java precommit dataflow examples tests to run on java 11

2020-03-02 Thread GitBox
pawelpasterz commented on a change in pull request #10882: Implement java precommit dataflow examples tests to run on java 11 URL: https://github.com/apache/beam/pull/10882#discussion_r386849053 ## File path: .test-infra/jenkins/job_PreCommit_Java_Examples_Dataflow_Java11.groovy

[GitHub] [beam] chadrik commented on issue #11022: [BEAM-7746] Resolve typing issues in filesystem

2020-03-02 Thread GitBox
chadrik commented on issue #11022: [BEAM-7746] Resolve typing issues in filesystem URL: https://github.com/apache/beam/pull/11022#issuecomment-593803091 R: @robertwb R: @udim This is an automated message from the

[GitHub] [beam] chadrik commented on a change in pull request #11022: [BEAM-7746] Resolve typing issues in filesystem

2020-03-02 Thread GitBox
chadrik commented on a change in pull request #11022: [BEAM-7746] Resolve typing issues in filesystem URL: https://github.com/apache/beam/pull/11022#discussion_r386809980 ## File path: sdks/python/apache_beam/io/localfilesystem.py ## @@ -136,10 +137,17 @@ def _path_open(

[GitHub] [beam] stale[bot] commented on issue #9899: [BEAM-8511] [WIP] KinesisIO.Read enhanced fanout

2020-03-02 Thread GitBox
stale[bot] commented on issue #9899: [BEAM-8511] [WIP] KinesisIO.Read enhanced fanout URL: https://github.com/apache/beam/pull/9899#issuecomment-593796235 This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If

[GitHub] [beam] Hannah-Jiang commented on issue #11023: [BEAM-9413]Fix post commit failure caused by docker migration

2020-03-02 Thread GitBox
Hannah-Jiang commented on issue #11023: [BEAM-9413]Fix post commit failure caused by docker migration URL: https://github.com/apache/beam/pull/11023#issuecomment-593789141 R: @tvalentyn Cc: @amaliujia , is it possible to cherrypick this PR to 2.20?

[GitHub] [beam] Hannah-Jiang commented on issue #11023: [BEAM-9413]Fix post commit failure caused by docker migration

2020-03-02 Thread GitBox
Hannah-Jiang commented on issue #11023: [BEAM-9413]Fix post commit failure caused by docker migration URL: https://github.com/apache/beam/pull/11023#issuecomment-593788560 Run Python Dataflow ValidatesContainer This is an

[GitHub] [beam] Hannah-Jiang opened a new pull request #11023: [BEAM-9413]Fix post commit failure caused by docker migration

2020-03-02 Thread GitBox
Hannah-Jiang opened a new pull request #11023: [BEAM-9413]Fix post commit failure caused by docker migration URL: https://github.com/apache/beam/pull/11023 **Please** add a meaningful description for your change here Thank you for your contribution!

[GitHub] [beam] chadrik commented on a change in pull request #11022: [BEAM-7746] Resolve typing issues in filesystem

2020-03-02 Thread GitBox
chadrik commented on a change in pull request #11022: [BEAM-7746] Resolve typing issues in filesystem URL: https://github.com/apache/beam/pull/11022#discussion_r386809980 ## File path: sdks/python/apache_beam/io/localfilesystem.py ## @@ -136,10 +137,17 @@ def _path_open(

[GitHub] [beam] chadrik commented on a change in pull request #11022: [BEAM-7746] Resolve typing issues in filesystem

2020-03-02 Thread GitBox
chadrik commented on a change in pull request #11022: [BEAM-7746] Resolve typing issues in filesystem URL: https://github.com/apache/beam/pull/11022#discussion_r386809980 ## File path: sdks/python/apache_beam/io/localfilesystem.py ## @@ -136,10 +137,17 @@ def _path_open(

[GitHub] [beam] chadrik commented on a change in pull request #11022: [BEAM-7746] Resolve typing issues in filesystem

2020-03-02 Thread GitBox
chadrik commented on a change in pull request #11022: [BEAM-7746] Resolve typing issues in filesystem URL: https://github.com/apache/beam/pull/11022#discussion_r386809980 ## File path: sdks/python/apache_beam/io/localfilesystem.py ## @@ -136,10 +137,17 @@ def _path_open(

[GitHub] [beam] chadrik opened a new pull request #11022: [BEAM-7746] Resolve typing issues in filesystem

2020-03-02 Thread GitBox
chadrik opened a new pull request #11022: [BEAM-7746] Resolve typing issues in filesystem URL: https://github.com/apache/beam/pull/11022 In order to properly type some of the complexities around the `filesystem` API I had to address 2 main problems: - properly type `CompressedFile`

[GitHub] [beam] robertwb commented on issue #11021: Remove excessive logging.

2020-03-02 Thread GitBox
robertwb commented on issue #11021: Remove excessive logging. URL: https://github.com/apache/beam/pull/11021#issuecomment-593764230 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to

[GitHub] [beam] robertwb commented on issue #11021: Remove excessive logging.

2020-03-02 Thread GitBox
robertwb commented on issue #11021: Remove excessive logging. URL: https://github.com/apache/beam/pull/11021#issuecomment-593764204 Run PythonLint PreCommit This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] robertwb commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

2020-03-02 Thread GitBox
robertwb commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. URL: https://github.com/apache/beam/pull/10915#discussion_r386794290 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@ -0,0

[GitHub] [beam] chadrik commented on issue #11007: [BEAM-7746] Support Timestamp/Duration equality testing with arbitrary objects

2020-03-02 Thread GitBox
chadrik commented on issue #11007: [BEAM-7746] Support Timestamp/Duration equality testing with arbitrary objects URL: https://github.com/apache/beam/pull/11007#issuecomment-593760530 R: @robertwb R: @udim This is

[GitHub] [beam] acrites commented on issue #10988: [BEAM-9382] Clean up of TestStreamTranscriptTests.

2020-03-02 Thread GitBox
acrites commented on issue #10988: [BEAM-9382] Clean up of TestStreamTranscriptTests. URL: https://github.com/apache/beam/pull/10988#issuecomment-593753721 Along the lines of what Kenn is saying, I had originally thought that these tests weren't really testing whether or not

[GitHub] [beam] lukecwik commented on issue #10897: [BEAM-2939] Java UnboundedSource SDF wrapper

2020-03-02 Thread GitBox
lukecwik commented on issue #10897: [BEAM-2939] Java UnboundedSource SDF wrapper URL: https://github.com/apache/beam/pull/10897#issuecomment-593745452 Tested internally within Google for UW and it passed. This is an automated

[GitHub] [beam] lukecwik merged pull request #10897: [BEAM-2939] Java UnboundedSource SDF wrapper

2020-03-02 Thread GitBox
lukecwik merged pull request #10897: [BEAM-2939] Java UnboundedSource SDF wrapper URL: https://github.com/apache/beam/pull/10897 This is an automated message from the Apache Git Service. To respond to the message, please

[beam] branch master updated (a29fdff -> bd7755f)

2020-03-02 Thread lcwik
This is an automated email from the ASF dual-hosted git repository. lcwik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from a29fdff [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. (#10915) add bd7755f [BEAM-2939]

[GitHub] [beam] kennknowles commented on issue #10988: [BEAM-9382] Clean up of TestStreamTranscriptTests.

2020-03-02 Thread GitBox
kennknowles commented on issue #10988: [BEAM-9382] Clean up of TestStreamTranscriptTests. URL: https://github.com/apache/beam/pull/10988#issuecomment-593740435 Since we are doing drive by comments, perhaps a clear comment describing exactly what the test is trying to verify would allow us

[GitHub] [beam] stale[bot] commented on issue #9942: [BEAM-3288] Do not drop data after trigger finishes

2020-03-02 Thread GitBox
stale[bot] commented on issue #9942: [BEAM-3288] Do not drop data after trigger finishes URL: https://github.com/apache/beam/pull/9942#issuecomment-593739632 This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs.

[GitHub] [beam] chamikaramj commented on issue #10951: [BEAM-8575] Modified the test to work for different runners.

2020-03-02 Thread GitBox
chamikaramj commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-593731682 Retest this please This is an automated message from

[GitHub] [beam] udim commented on issue #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

2020-03-02 Thread GitBox
udim commented on issue #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. URL: https://github.com/apache/beam/pull/10915#issuecomment-593731439 I believe this PR broke precommits. Please take care to run tests on Jenkins before merging. ``` > Task

[GitHub] [beam] chunyang commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
chunyang commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593729608 I am able to run the integration test `apache_beam.io.gcp.bigquery_file_loads_test:BigQueryFileLoadsIT` but for

[GitHub] [beam] bumblebee-coming commented on issue #10951: [BEAM-8575] Modified the test to work for different runners.

2020-03-02 Thread GitBox
bumblebee-coming commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-593729091 I first added a new matcher to test there are any number of 15s. Later I realized I can partition the PCollection

[GitHub] [beam] chunyang commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
chunyang commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593722179 I think I need to fix a few of the integration tests that don't provide a schema or use `SCHEMA_AUTODETECT`.

[GitHub] [beam] chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook

2020-03-02 Thread GitBox
chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook URL: https://github.com/apache/beam/pull/10810#issuecomment-593720763 Run Python PreCommit This is an automated message from the

[GitHub] [beam] chamikaramj commented on issue #11018: [BEAM-9415] fix postcommit xvr tests

2020-03-02 Thread GitBox
chamikaramj commented on issue #11018: [BEAM-9415] fix postcommit xvr tests URL: https://github.com/apache/beam/pull/11018#issuecomment-593720543 Run Python PreCommit This is an automated message from the Apache Git Service.

[GitHub] [beam] udim commented on issue #11016: Reduce warnings in pytest runs.

2020-03-02 Thread GitBox
udim commented on issue #11016: Reduce warnings in pytest runs. URL: https://github.com/apache/beam/pull/11016#issuecomment-593719479 Run Python PreCommit This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK

2020-03-02 Thread GitBox
kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK URL: https://github.com/apache/beam/pull/10949#discussion_r386745270 ## File path:

[GitHub] [beam] kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK

2020-03-02 Thread GitBox
kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK URL: https://github.com/apache/beam/pull/10949#discussion_r386744848 ## File path:

[GitHub] [beam] kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK

2020-03-02 Thread GitBox
kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK URL: https://github.com/apache/beam/pull/10949#discussion_r386743300 ## File path:

[GitHub] [beam] kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK

2020-03-02 Thread GitBox
kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK URL: https://github.com/apache/beam/pull/10949#discussion_r386745742 ## File path:

[GitHub] [beam] kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK

2020-03-02 Thread GitBox
kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK URL: https://github.com/apache/beam/pull/10949#discussion_r386740644 ## File path:

[GitHub] [beam] kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK

2020-03-02 Thread GitBox
kennknowles commented on a change in pull request #10949: [BEAM-9371] Add SideInputLoadTest to Java SDK URL: https://github.com/apache/beam/pull/10949#discussion_r386743669 ## File path:

[GitHub] [beam] robertwb commented on issue #11021: Remove excessive logging.

2020-03-02 Thread GitBox
robertwb commented on issue #11021: Remove excessive logging. URL: https://github.com/apache/beam/pull/11021#issuecomment-593713484 R: @ruwang may be a candidate for a cherry-pick This is an automated message from the Apache

[GitHub] [beam] robertwb opened a new pull request #11021: Remove excessive logging.

2020-03-02 Thread GitBox
robertwb opened a new pull request #11021: Remove excessive logging. URL: https://github.com/apache/beam/pull/11021 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ]

[GitHub] [beam] rohdesamuel commented on issue #11005: [BEAM-8335] Modify the StreamingCache to subclass the CacheManager

2020-03-02 Thread GitBox
rohdesamuel commented on issue #11005: [BEAM-8335] Modify the StreamingCache to subclass the CacheManager URL: https://github.com/apache/beam/pull/11005#issuecomment-593703837 R: @robertwb This is an automated message from

[GitHub] [beam] pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593703663 Run Python 3.7 PostCommit This is an automated

[GitHub] [beam] KevinGG commented on issue #11020: [BEAM-7926] Update Data Visualization

2020-03-02 Thread GitBox
KevinGG commented on issue #11020: [BEAM-7926] Update Data Visualization URL: https://github.com/apache/beam/pull/11020#issuecomment-593701514 R: @rohdesamuel R: @pabloem PTAL, thanks! This is an automated message

[GitHub] [beam] pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593700905 Run Python 3.7 PostCommit This is an automated

[GitHub] [beam] rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream URL: https://github.com/apache/beam/pull/10497#discussion_r386726726 ## File path: sdks/python/apache_beam/runners/direct/transform_evaluator.py ## @@ -471,7 +517,14 @@ def

[GitHub] [beam] rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream URL: https://github.com/apache/beam/pull/10497#discussion_r386726510 ## File path: sdks/python/apache_beam/runners/direct/transform_evaluator.py ## @@ -471,7 +517,14 @@ def

[GitHub] [beam] pabloem commented on issue #11019: Reducing the number of API calls to BQ table.get

2020-03-02 Thread GitBox
pabloem commented on issue #11019: Reducing the number of API calls to BQ table.get URL: https://github.com/apache/beam/pull/11019#issuecomment-593693425 Run Python 3.7 PostCommit This is an automated message from the Apache

[GitHub] [beam] KevinGG opened a new pull request #11020: [BEAM-7926] Update Data Visualization

2020-03-02 Thread GitBox
KevinGG opened a new pull request #11020: [BEAM-7926] Update Data Visualization URL: https://github.com/apache/beam/pull/11020 1. Added include_window_info and visualize_data as **kwargs passed into `show`. 2. Updated javascripts to make the data visualization smooth and

[GitHub] [beam] chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook

2020-03-02 Thread GitBox
chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook URL: https://github.com/apache/beam/pull/10810#issuecomment-593691076 Run Python PreCommit This is an automated message from the

[GitHub] [beam] robertwb merged pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

2020-03-02 Thread GitBox
robertwb merged pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. URL: https://github.com/apache/beam/pull/10915 This is an automated message from the Apache Git Service. To respond

[beam] branch master updated (a167255 -> a29fdff)

2020-03-02 Thread robertwb
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from a167255 Merge pull request #10989 from lukecwik/beam9397 add a29fdff [BEAM-8335] Add PCollection to DataFrame

[GitHub] [beam] rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream URL: https://github.com/apache/beam/pull/10497#discussion_r386723318 ## File path: sdks/python/apache_beam/testing/test_stream.py ## @@ -314,3 +355,239 @@ def

[GitHub] [beam] rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream URL: https://github.com/apache/beam/pull/10497#discussion_r386723371 ## File path: sdks/python/apache_beam/testing/test_stream.py ## @@ -314,3 +355,239 @@ def

[GitHub] [beam] pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593679995 retest this please This is an automated message

[GitHub] [beam] pabloem commented on issue #11019: Reducing the number of API calls to BQ table.get

2020-03-02 Thread GitBox
pabloem commented on issue #11019: Reducing the number of API calls to BQ table.get URL: https://github.com/apache/beam/pull/11019#issuecomment-593679725 Run Python 3.7 PostCommit This is an automated message from the Apache

[GitHub] [beam] pabloem commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream

2020-03-02 Thread GitBox
pabloem commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream URL: https://github.com/apache/beam/pull/10497#discussion_r386714777 ## File path: sdks/python/apache_beam/runners/direct/transform_evaluator.py ## @@ -471,7 +517,14 @@ def

[GitHub] [beam] rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream URL: https://github.com/apache/beam/pull/10497#discussion_r386711043 ## File path: sdks/python/apache_beam/testing/test_stream.py ## @@ -314,3 +355,239 @@ def

[GitHub] [beam] chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook

2020-03-02 Thread GitBox
chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook URL: https://github.com/apache/beam/pull/10810#issuecomment-593672795 Run Python2_PVR_Flink PreCommit This is an automated

[GitHub] [beam] lukecwik commented on issue #10988: [BEAM-9382] Clean up of TestStreamTranscriptTests.

2020-03-02 Thread GitBox
lukecwik commented on issue #10988: [BEAM-9382] Clean up of TestStreamTranscriptTests. URL: https://github.com/apache/beam/pull/10988#issuecomment-593667609 Drive by comment: Shouldn't we disable the test for the direct runner if it isn't supported and file a bug instead of editing

[GitHub] [beam] chamikaramj commented on issue #10477: [BEAM-8932][Cleanup] Cleanup pubsubio by removing optionality and adding defaults to builders.

2020-03-02 Thread GitBox
chamikaramj commented on issue #10477: [BEAM-8932][Cleanup] Cleanup pubsubio by removing optionality and adding defaults to builders. URL: https://github.com/apache/beam/pull/10477#issuecomment-593662732 Run Beam PostCommit

[GitHub] [beam] chamikaramj commented on issue #10477: [BEAM-8932][Cleanup] Cleanup pubsubio by removing optionality and adding defaults to builders.

2020-03-02 Thread GitBox
chamikaramj commented on issue #10477: [BEAM-8932][Cleanup] Cleanup pubsubio by removing optionality and adding defaults to builders. URL: https://github.com/apache/beam/pull/10477#issuecomment-593662585 LGTM. Thanks. This

[GitHub] [beam] chamikaramj commented on issue #10477: [BEAM-8932][Cleanup] Cleanup pubsubio by removing optionality and adding defaults to builders.

2020-03-02 Thread GitBox
chamikaramj commented on issue #10477: [BEAM-8932][Cleanup] Cleanup pubsubio by removing optionality and adding defaults to builders. URL: https://github.com/apache/beam/pull/10477#issuecomment-593662682 Run Dataflow ValidatesRunner

[GitHub] [beam] chamikaramj commented on issue #10477: [BEAM-8932][Cleanup] Cleanup pubsubio by removing optionality and adding defaults to builders.

2020-03-02 Thread GitBox
chamikaramj commented on issue #10477: [BEAM-8932][Cleanup] Cleanup pubsubio by removing optionality and adding defaults to builders. URL: https://github.com/apache/beam/pull/10477#issuecomment-593662632 Retest this please

[GitHub] [beam] rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream URL: https://github.com/apache/beam/pull/10497#discussion_r386694935 ## File path: sdks/python/apache_beam/runners/direct/transform_evaluator.py ## @@ -421,8 +424,12 @@ def

[GitHub] [beam] rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream URL: https://github.com/apache/beam/pull/10497#discussion_r386694347 ## File path: sdks/python/apache_beam/runners/direct/transform_evaluator.py ## @@ -432,6 +439,45 @@ def

[GitHub] [beam] ihji commented on a change in pull request #11018: [BEAM-9415] fix postcommit xvr tests

2020-03-02 Thread GitBox
ihji commented on a change in pull request #11018: [BEAM-9415] fix postcommit xvr tests URL: https://github.com/apache/beam/pull/11018#discussion_r386693638 ## File path: sdks/java/testing/expansion-service/build.gradle ## @@ -28,7 +28,7 @@ dependencies { compile

[GitHub] [beam] rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10497: [BEAM-8335] Add the ReverseTestStream URL: https://github.com/apache/beam/pull/10497#discussion_r386693162 ## File path: sdks/python/apache_beam/testing/test_stream.py ## @@ -314,3 +355,239 @@ def

[GitHub] [beam] chamikaramj commented on a change in pull request #11018: [BEAM-9415] fix postcommit xvr tests

2020-03-02 Thread GitBox
chamikaramj commented on a change in pull request #11018: [BEAM-9415] fix postcommit xvr tests URL: https://github.com/apache/beam/pull/11018#discussion_r386692400 ## File path: sdks/java/testing/expansion-service/build.gradle ## @@ -28,7 +28,7 @@ dependencies {

[GitHub] [beam] pabloem commented on issue #11019: Reducing the number of API calls to BQ table.get

2020-03-02 Thread GitBox
pabloem commented on issue #11019: Reducing the number of API calls to BQ table.get URL: https://github.com/apache/beam/pull/11019#issuecomment-593657783 Run Python 3.7 PostCommit This is an automated message from the Apache

[GitHub] [beam] pabloem opened a new pull request #11019: Reducing the number of API calls to BQ table.get

2020-03-02 Thread GitBox
pabloem opened a new pull request #11019: Reducing the number of API calls to BQ table.get URL: https://github.com/apache/beam/pull/11019 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this

[GitHub] [beam] lukecwik merged pull request #10989: [BEAM-9397] Pass all supported StartBundleContext/FinishBundleContext except output receiver parameters to start bundle/finish bundle methods.

2020-03-02 Thread GitBox
lukecwik merged pull request #10989: [BEAM-9397] Pass all supported StartBundleContext/FinishBundleContext except output receiver parameters to start bundle/finish bundle methods. URL: https://github.com/apache/beam/pull/10989

[beam] branch master updated: [BEAM-9397] Pass all but output receiver parameters to start bundle/finish bundle methods.

2020-03-02 Thread lcwik
This is an automated email from the ASF dual-hosted git repository. lcwik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 360c4fe [BEAM-9397] Pass all but output receiver

[GitHub] [beam] chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook

2020-03-02 Thread GitBox
chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook URL: https://github.com/apache/beam/pull/10810#issuecomment-593654495 >> If it's opt-out, what happens when the user doesn't set up their local repo (per the instructions above)? > Nothing.

[GitHub] [beam] chadrik commented on a change in pull request #10810: [BEAM-9274] Support running yapf in a git pre-commit hook

2020-03-02 Thread GitBox
chadrik commented on a change in pull request #10810: [BEAM-9274] Support running yapf in a git pre-commit hook URL: https://github.com/apache/beam/pull/10810#discussion_r386686435 ## File path: .pre-commit-config.yaml ## @@ -0,0 +1,32 @@ +# Licensed under the Apache

[GitHub] [beam] ihji commented on issue #11018: [BEAM-9415] fix postcommit xvr tests

2020-03-02 Thread GitBox
ihji commented on issue #11018: [BEAM-9415] fix postcommit xvr tests URL: https://github.com/apache/beam/pull/11018#issuecomment-593646636 Updated. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593643579 restest this please This is an automated message

[beam] branch master updated (06b3111 -> 6b4b99c)

2020-03-02 Thread pabloem
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 06b3111 Merge pull request #11014: [BEAM-8925] Tika version update to 1.23 add 6b4b99c Merge pull request

[GitHub] [beam] pabloem merged pull request #10968: [BEAM-9381] Adding display data to BoundedSource SDF

2020-03-02 Thread GitBox
pabloem merged pull request #10968: [BEAM-9381] Adding display data to BoundedSource SDF URL: https://github.com/apache/beam/pull/10968 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] chadrik commented on issue #10734: [BEAM-8979] reintroduce mypy-protobuf stub generation

2020-03-02 Thread GitBox
chadrik commented on issue #10734: [BEAM-8979] reintroduce mypy-protobuf stub generation URL: https://github.com/apache/beam/pull/10734#issuecomment-593633730 Oh, You have to create your own venv and install build-requirements.txt into it. Yeah, pyproject.toml would solve this.

[GitHub] [beam] udim commented on issue #10734: [BEAM-8979] reintroduce mypy-protobuf stub generation

2020-03-02 Thread GitBox
udim commented on issue #10734: [BEAM-8979] reintroduce mypy-protobuf stub generation URL: https://github.com/apache/beam/pull/10734#issuecomment-593632171 It works through the Gradle tasks' virtualenvs, so Jenkins is fine. See bug for workaround/solution. Perhaps using pyproject.toml is

[GitHub] [beam] chadrik commented on issue #10734: [BEAM-8979] reintroduce mypy-protobuf stub generation

2020-03-02 Thread GitBox
chadrik commented on issue #10734: [BEAM-8979] reintroduce mypy-protobuf stub generation URL: https://github.com/apache/beam/pull/10734#issuecomment-593625714 Arrg! This was working! Right? Do you have any idea if something changed?

[GitHub] [beam] chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook

2020-03-02 Thread GitBox
chadrik commented on issue #10810: [BEAM-9274] Support running yapf in a git pre-commit hook URL: https://github.com/apache/beam/pull/10810#issuecomment-593623997 > Is this opt-in? completely opt-in. >If it's opt-out, what happens when the user doesn't set up their local

[GitHub] [beam] chamikaramj commented on issue #11018: [BEAM-9415] fix postcommit xvr tests

2020-03-02 Thread GitBox
chamikaramj commented on issue #11018: [BEAM-9415] fix postcommit xvr tests URL: https://github.com/apache/beam/pull/11018#issuecomment-593622896 Could you add more details about the breakage and the fix to the PR description and JIRA ? (since we are updating External.java here).

[GitHub] [beam] rohdesamuel commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. URL: https://github.com/apache/beam/pull/10915#discussion_r386646890 ## File path: sdks/python/apache_beam/runners/interactive/utils_test.py ##

[GitHub] [beam] rohdesamuel commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. URL: https://github.com/apache/beam/pull/10915#discussion_r386646690 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@

[GitHub] [beam] rohdesamuel commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

2020-03-02 Thread GitBox
rohdesamuel commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. URL: https://github.com/apache/beam/pull/10915#discussion_r386646757 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@

[GitHub] [beam] ihji commented on issue #10621: [BEAM-9056] Staging artifacts from environment

2020-03-02 Thread GitBox
ihji commented on issue #10621: [BEAM-9056] Staging artifacts from environment URL: https://github.com/apache/beam/pull/10621#issuecomment-593616556 @chamikaramj @robertwb Comments addressed. PTAL. Thanks! This is an

[GitHub] [beam] ihji commented on a change in pull request #10621: [BEAM-9056] Staging artifacts from environment

2020-03-02 Thread GitBox
ihji commented on a change in pull request #10621: [BEAM-9056] Staging artifacts from environment URL: https://github.com/apache/beam/pull/10621#discussion_r386643699 ## File path:

[GitHub] [beam] ihji commented on issue #11018: [BEAM-9415] fix postcommit xvr tests

2020-03-02 Thread GitBox
ihji commented on issue #11018: [BEAM-9415] fix postcommit xvr tests URL: https://github.com/apache/beam/pull/11018#issuecomment-593614770 R: @chamikaramj This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] ihji opened a new pull request #11018: [BEAM-9415] fix postcommit xvr tests

2020-03-02 Thread GitBox
ihji opened a new pull request #11018: [BEAM-9415] fix postcommit xvr tests URL: https://github.com/apache/beam/pull/11018 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ]

[GitHub] [beam] Hk-tang opened a new pull request #11017: Changed to using StandardCharsets.UTF_8

2020-03-02 Thread GitBox
Hk-tang opened a new pull request #11017: Changed to using StandardCharsets.UTF_8 URL: https://github.com/apache/beam/pull/11017 Using new String(bytes, StandardCharsets.UTF_8) avoids having to catch the UnsupportedEncodingException Thank you for

[GitHub] [beam] pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593609967 jenkins is the worst : ) This is an automated

[GitHub] [beam] pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593609918 restest this please This is an automated message

[GitHub] [beam] pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
pabloem commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593606703 Retest this please This is an automated message

[GitHub] [beam] pabloem commented on issue #10968: [BEAM-9381] Adding display data to BoundedSource SDF

2020-03-02 Thread GitBox
pabloem commented on issue #10968: [BEAM-9381] Adding display data to BoundedSource SDF URL: https://github.com/apache/beam/pull/10968#issuecomment-593607043 Run Python PreCommit This is an automated message from the Apache

[GitHub] [beam] chunyang commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK

2020-03-02 Thread GitBox
chunyang commented on issue #10979: [BEAM-8841] Support writing data to BigQuery via Avro in Python SDK URL: https://github.com/apache/beam/pull/10979#issuecomment-593605041 retest this please This is an automated message

[GitHub] [beam] alexvanboxel commented on issue #10413: [BEAM-9035] Typed options for Row Schema and Field

2020-03-02 Thread GitBox
alexvanboxel commented on issue #10413: [BEAM-9035] Typed options for Row Schema and Field URL: https://github.com/apache/beam/pull/10413#issuecomment-593591725 Verified that the failed Python 2 build isn't due to the PR. This PR is ready for review. I have 2 PR's based on this one ready:

[GitHub] [beam] ibzib commented on a change in pull request #11011: [website] Update link to environment_type (SDK harness configuration)

2020-03-02 Thread GitBox
ibzib commented on a change in pull request #11011: [website] Update link to environment_type (SDK harness configuration) URL: https://github.com/apache/beam/pull/11011#discussion_r386614175 ## File path: website/src/roadmap/portability.md ## @@ -165,7 +165,3 @@ Please

[GitHub] [beam] udim opened a new pull request #11016: Reduce warnings in pytest runs.

2020-03-02 Thread GitBox
udim opened a new pull request #11016: Reduce warnings in pytest runs. URL: https://github.com/apache/beam/pull/11016 From 2233 down to 412. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and

[GitHub] [beam] udim commented on issue #10822: [BEAM-7746] Minor typing updates / fixes

2020-03-02 Thread GitBox
udim commented on issue #10822: [BEAM-7746] Minor typing updates / fixes URL: https://github.com/apache/beam/pull/10822#issuecomment-593584144 I want to leave some commits and squash others. There is no way to do that in the GH UI.

[GitHub] [beam] robertwb commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

2020-03-02 Thread GitBox
robertwb commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. URL: https://github.com/apache/beam/pull/10915#discussion_r386591818 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@ -0,0

[GitHub] [beam] robertwb commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

2020-03-02 Thread GitBox
robertwb commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. URL: https://github.com/apache/beam/pull/10915#discussion_r386592005 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@ -0,0

[GitHub] [beam] robertwb commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.

2020-03-02 Thread GitBox
robertwb commented on a change in pull request #10915: [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner. URL: https://github.com/apache/beam/pull/10915#discussion_r386592264 ## File path: sdks/python/apache_beam/runners/interactive/utils_test.py ## @@

  1   2   >