[GitHub] [beam] aromanenko-dev commented on pull request #11459: [BEAM-2546] Add InfluxDbIO

2020-08-04 Thread GitBox
aromanenko-dev commented on pull request #11459: URL: https://github.com/apache/beam/pull/11459#issuecomment-668588914 @bipinupd Kind ping on this PR, ptal on my comments This is an automated message from the Apache Git

[GitHub] [beam] piotr-szuberski commented on pull request #12325: [BEAM-10543] Upgrade Kafka cross-language python tests

2020-08-04 Thread GitBox
piotr-szuberski commented on pull request #12325: URL: https://github.com/apache/beam/pull/12325#issuecomment-668550585 Run Python 2 PostCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] kkucharc commented on pull request #12435: [WIP] [BEAM-10616] Added Python Pardo load tests for streaming on Dataflow

2020-08-04 Thread GitBox
kkucharc commented on pull request #12435: URL: https://github.com/apache/beam/pull/12435#issuecomment-668546700 Run Python Load Tests ParDo Dataflow Streaming This is an automated message from the Apache Git Service. To

[GitHub] [beam] kkucharc commented on pull request #12435: [WIP] [BEAM-10616] Added Python Pardo load tests for streaming on Dataflow

2020-08-04 Thread GitBox
kkucharc commented on pull request #12435: URL: https://github.com/apache/beam/pull/12435#issuecomment-668543971 run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] mxm commented on a change in pull request #12455: [BEAM-10630] Include data from load tests in the release process

2020-08-04 Thread GitBox
mxm commented on a change in pull request #12455: URL: https://github.com/apache/beam/pull/12455#discussion_r464971852 ## File path: website/www/site/content/en/contribute/release-guide.md ## @@ -244,7 +247,21 @@ __Attention__: Only PMC has permission to perform this. If you

[GitHub] [beam] aromanenko-dev merged pull request #12397: [BEAM-7632] Add Python quickstart instructions for Flink and Spark.

2020-08-04 Thread GitBox
aromanenko-dev merged pull request #12397: URL: https://github.com/apache/beam/pull/12397 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [beam] piotr-szuberski commented on pull request #12325: [BEAM-10543] Upgrade Kafka cross-language python tests

2020-08-04 Thread GitBox
piotr-szuberski commented on pull request #12325: URL: https://github.com/apache/beam/pull/12325#issuecomment-668513867 Run Python 2 PostCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] piotr-szuberski commented on pull request #12325: [BEAM-10543] Upgrade Kafka cross-language python tests

2020-08-04 Thread GitBox
piotr-szuberski commented on pull request #12325: URL: https://github.com/apache/beam/pull/12325#issuecomment-668513966 Run Python 3.7 PostCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] piotr-szuberski commented on pull request #12325: [BEAM-10543] Upgrade Kafka cross-language python tests

2020-08-04 Thread GitBox
piotr-szuberski commented on pull request #12325: URL: https://github.com/apache/beam/pull/12325#issuecomment-668488102 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [beam] mxm merged pull request #12449: [BEAM-10622] Prefix Gradle paths with a colon for user-facing output

2020-08-04 Thread GitBox
mxm merged pull request #12449: URL: https://github.com/apache/beam/pull/12449 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [beam] piotr-szuberski edited a comment on pull request #11181: [BEAM-9500] [WIP] Refactor load tests

2020-08-04 Thread GitBox
piotr-szuberski edited a comment on pull request #11181: URL: https://github.com/apache/beam/pull/11181#issuecomment-668478754 > @piotr-szuberski - what is the next step for this PR? Is it still active? Should we close it? Sorry for such delay, I totally forgot to response. I'll

[GitHub] [beam] piotr-szuberski commented on pull request #11181: [BEAM-9500] [WIP] Refactor load tests

2020-08-04 Thread GitBox
piotr-szuberski commented on pull request #11181: URL: https://github.com/apache/beam/pull/11181#issuecomment-668478754 > @piotr-szuberski - what is the next step for this PR? Is it still active? Should we close it?

[GitHub] [beam] piotr-szuberski closed pull request #11181: [BEAM-9500] [WIP] Refactor load tests

2020-08-04 Thread GitBox
piotr-szuberski closed pull request #11181: URL: https://github.com/apache/beam/pull/11181 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [beam] piotr-szuberski commented on pull request #12325: [BEAM-10543] Upgrade Kafka cross-language python tests

2020-08-04 Thread GitBox
piotr-szuberski commented on pull request #12325: URL: https://github.com/apache/beam/pull/12325#issuecomment-668472410 Run Python 2 PostCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] piotr-szuberski commented on pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on pull request #12422: URL: https://github.com/apache/beam/pull/12422#issuecomment-668470214 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] piotr-szuberski commented on pull request #12325: [BEAM-10543] Upgrade Kafka cross-language python tests

2020-08-04 Thread GitBox
piotr-szuberski commented on pull request #12325: URL: https://github.com/apache/beam/pull/12325#issuecomment-668469132 @ihji @chamikaramj ping > Thanks, looks good to me overall. > > Is there any way to share the pipeline codes in `CrossLanguageKafkaIOTest` and

[GitHub] [beam] piotr-szuberski edited a comment on pull request #12325: [BEAM-10543] Upgrade Kafka cross-language python tests

2020-08-04 Thread GitBox
piotr-szuberski edited a comment on pull request #12325: URL: https://github.com/apache/beam/pull/12325#issuecomment-668469132 > Is there any way to share the pipeline codes in `CrossLanguageKafkaIOTest` and `CrossLanguageKafkaIO`? Could we share the pipeline codes and use

[GitHub] [beam] ihji commented on a change in pull request #12403: [BEAM-10597] Propagate BigQuery streaming insert throttled time to Dataflow worker

2020-08-04 Thread GitBox
ihji commented on a change in pull request #12403: URL: https://github.com/apache/beam/pull/12403#discussion_r464894202 ## File path: runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/BatchModeExecutionContext.java ## @@ -543,6

[GitHub] [beam] slawomir-andrian commented on pull request #12296: [BEAM-10522] Added SnowflakeIO connector guide

2020-08-04 Thread GitBox
slawomir-andrian commented on pull request #12296: URL: https://github.com/apache/beam/pull/12296#issuecomment-668447132 @TheNeuralBit Have you had a chance to take a look? :) This is an automated message from the Apache

[GitHub] [beam] piotr-szuberski commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464840886 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -35,33 +38,45 @@ import

[GitHub] [beam] piotr-szuberski commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464853325 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -99,28 +116,101 @@ private void runRead()

[GitHub] [beam] davidak09 commented on a change in pull request #12050: [BEAM-10292] DefaultFilenamePolicy.ParamsCoder uses ResourceIdCoder

2020-08-04 Thread GitBox
davidak09 commented on a change in pull request #12050: URL: https://github.com/apache/beam/pull/12050#discussion_r464849600 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/DefaultFilenamePolicy.java ## @@ -182,19 +184,26 @@ public void encode(Params value,

[GitHub] [beam] mwalenia commented on pull request #12453: [BEAM-9421] Add Java snippets to NLP documentation.

2020-08-04 Thread GitBox
mwalenia commented on pull request #12453: URL: https://github.com/apache/beam/pull/12453#issuecomment-668424165 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] piotr-szuberski commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464846401 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -99,28 +116,101 @@ private void runRead()

[GitHub] [beam] piotr-szuberski commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464846401 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -99,28 +116,101 @@ private void runRead()

[GitHub] [beam] piotr-szuberski commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464842213 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -99,28 +116,101 @@ private void runRead()

[GitHub] [beam] piotr-szuberski commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464842213 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -99,28 +116,101 @@ private void runRead()

[GitHub] [beam] piotr-szuberski commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464842213 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -99,28 +116,101 @@ private void runRead()

[GitHub] [beam] piotr-szuberski commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464841242 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -99,28 +116,101 @@ private void runRead()

[GitHub] [beam] piotr-szuberski commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-04 Thread GitBox
piotr-szuberski commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464840886 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -35,33 +38,45 @@ import

[GitHub] [beam] kanterov commented on a change in pull request #10413: [BEAM-9035] Typed options for Row Schema and Field

2020-08-04 Thread GitBox
kanterov commented on a change in pull request #10413: URL: https://github.com/apache/beam/pull/10413#discussion_r464824602 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/Schema.java ## @@ -986,10 +1246,8 @@ public Field getField(String name) { /**

[GitHub] [beam] pabloem commented on pull request #12203: [BEAM-6928] Make Python SDK custom Sink the default Sink for BigQuery

2020-08-03 Thread GitBox
pabloem commented on pull request #12203: URL: https://github.com/apache/beam/pull/12203#issuecomment-668367941 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] pabloem commented on pull request #12203: [BEAM-6928] Make Python SDK custom Sink the default Sink for BigQuery

2020-08-03 Thread GitBox
pabloem commented on pull request #12203: URL: https://github.com/apache/beam/pull/12203#issuecomment-668367716 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] ihji commented on pull request #12325: [BEAM-10543] Upgrade Kafka cross-language python tests

2020-08-03 Thread GitBox
ihji commented on pull request #12325: URL: https://github.com/apache/beam/pull/12325#issuecomment-668359376 Run Python 3.7 PostCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] ihji commented on pull request #12325: [BEAM-10543] Upgrade Kafka cross-language python tests

2020-08-03 Thread GitBox
ihji commented on pull request #12325: URL: https://github.com/apache/beam/pull/12325#issuecomment-668359113 Thanks, looks good to me overall. Is there any way to share the pipeline codes in `CrossLanguageKafkaIOTest` and `CrossLanguageKafkaIO`? Could we share the pipeline codes and

[GitHub] [beam] amaliujia commented on pull request #12461: [BEAM-10633] UdfImpl should support return java.util.List

2020-08-03 Thread GitBox
amaliujia commented on pull request #12461: URL: https://github.com/apache/beam/pull/12461#issuecomment-668356948 R: @ibzib This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] amaliujia opened a new pull request #12461: [BEAM-10633] UdfImpl should support return java.util.List

2020-08-03 Thread GitBox
amaliujia opened a new pull request #12461: URL: https://github.com/apache/beam/pull/12461 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose

[GitHub] [beam] damondouglas commented on a change in pull request #12448: [BEAM-9679] Add Additional Parameters lesson to Go SDK Katas

2020-08-03 Thread GitBox
damondouglas commented on a change in pull request #12448: URL: https://github.com/apache/beam/pull/12448#discussion_r464765533 ## File path: learning/katas/go/core_transforms/additional_parameters/additional_parameters/task.md ## @@ -0,0 +1,84 @@ + + +# Additional Parameters

[GitHub] [beam] KevinGG opened a new pull request #12460: [BEAM-10545] HtmlView module

2020-08-03 Thread GitBox
KevinGG opened a new pull request #12460: URL: https://github.com/apache/beam/pull/12460 1. Added a HtmlView module to render given HTML and execute given scripts from the provider model. 2. Integrated react framework to the jest testing framework and eslint. 3. Added line

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12426: [BEAM-7996] Add support for MapType and Nulls in container types for Python RowCoder

2020-08-03 Thread GitBox
TheNeuralBit commented on a change in pull request #12426: URL: https://github.com/apache/beam/pull/12426#discussion_r464732307 ## File path: model/pipeline/src/main/proto/beam_runner_api.proto ## @@ -855,10 +855,21 @@ message StandardCoders { // BOOLEAN:

[GitHub] [beam] TheNeuralBit commented on pull request #12454: [BEAM-10629] KnownBuilderInstances in ExternalTransformRegistrar

2020-08-03 Thread GitBox
TheNeuralBit commented on pull request #12454: URL: https://github.com/apache/beam/pull/12454#issuecomment-668302572 Oh also it looks like the Java PreCommit failure may be due to a legitimate issue in spotbugs, you may want to check that locally.

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12454: [BEAM-10629] KnownBuilderInstances in ExternalTransformRegistrar

2020-08-03 Thread GitBox
TheNeuralBit commented on a change in pull request #12454: URL: https://github.com/apache/beam/pull/12454#discussion_r464726158 ## File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionService.java ## @@ -107,39 +107,39 @@

[GitHub] [beam] youngoli commented on pull request #12350: [BEAM-10289] Dynamic splitting implementation.

2020-08-03 Thread GitBox
youngoli commented on pull request #12350: URL: https://github.com/apache/beam/pull/12350#issuecomment-668301428 Run Go PostCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] lostluck commented on a change in pull request #12445: [BEAM-9919] Added an External Transform API to Go SDK

2020-08-03 Thread GitBox
lostluck commented on a change in pull request #12445: URL: https://github.com/apache/beam/pull/12445#discussion_r464717996 ## File path: sdks/go/examples/xlang/wordcount/xlang_wordcount.go ## @@ -0,0 +1,100 @@ +package main + +import ( + "context" + "flag" +

[GitHub] [beam] KevinGG commented on a change in pull request #12452: [BEAM-10623] Add workflow to run Beam python tests on Linux/Windows/Mac platforms

2020-08-03 Thread GitBox
KevinGG commented on a change in pull request #12452: URL: https://github.com/apache/beam/pull/12452#discussion_r464721910 ## File path: sdks/python/apache_beam/runners/interactive/interactive_runner_test.py ## @@ -66,6 +66,7 @@ class InteractiveRunnerTest(unittest.TestCase):

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12452: [BEAM-10623] Add workflow to run Beam python tests on Linux/Windows/Mac platforms

2020-08-03 Thread GitBox
TheNeuralBit commented on a change in pull request #12452: URL: https://github.com/apache/beam/pull/12452#discussion_r464721198 ## File path: sdks/python/apache_beam/testing/datatype_inference_test.py ## @@ -174,6 +175,7 @@ def test_infer_typehints_schema(self, _, data,

[GitHub] [beam] aaltay merged pull request #12304: [BEAM-10499] Adds a descriptive toString to SamzaRunner KeyedTimerData

2020-08-03 Thread GitBox
aaltay merged pull request #12304: URL: https://github.com/apache/beam/pull/12304 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] aaltay commented on pull request #12304: [BEAM-10499] Adds a descriptive toString to SamzaRunner KeyedTimerData

2020-08-03 Thread GitBox
aaltay commented on pull request #12304: URL: https://github.com/apache/beam/pull/12304#issuecomment-668292310 Merging. @kennknowles @xinyuiscool - if you have additional comments please fix it with a new pr. This is an

[GitHub] [beam] aaltay commented on a change in pull request #12452: [BEAM-10623] Add workflow to run Beam python tests on Linux/Windows/Mac platforms

2020-08-03 Thread GitBox
aaltay commented on a change in pull request #12452: URL: https://github.com/apache/beam/pull/12452#discussion_r464716197 ## File path: sdks/python/apache_beam/runners/portability/portable_runner_test.py ## @@ -284,6 +284,7 @@ def create_options(self): return options

[GitHub] [beam] TheNeuralBit merged pull request #12456: [BEAM-10631] Fix performance of Schema#indexOf

2020-08-03 Thread GitBox
TheNeuralBit merged pull request #12456: URL: https://github.com/apache/beam/pull/12456 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [beam] danielxjd commented on a change in pull request #12223: [Beam-4379] Make ParquetIO read splittable

2020-08-03 Thread GitBox
danielxjd commented on a change in pull request #12223: URL: https://github.com/apache/beam/pull/12223#discussion_r464705615 ## File path: sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java ## @@ -230,12 +281,271 @@ public ReadFiles

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12422: [BEAM-601] Run KinesisIOIT with localstack

2020-08-03 Thread GitBox
TheNeuralBit commented on a change in pull request #12422: URL: https://github.com/apache/beam/pull/12422#discussion_r464702259 ## File path: sdks/java/io/kinesis/src/test/java/org/apache/beam/sdk/io/kinesis/KinesisIOIT.java ## @@ -99,28 +116,101 @@ private void runRead() {

[GitHub] [beam] danielxjd commented on a change in pull request #12223: [Beam-4379] Make ParquetIO read splittable

2020-08-03 Thread GitBox
danielxjd commented on a change in pull request #12223: URL: https://github.com/apache/beam/pull/12223#discussion_r464703969 ## File path: sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java ## @@ -230,12 +281,271 @@ public ReadFiles

[GitHub] [beam] danielxjd commented on a change in pull request #12223: [Beam-4379] Make ParquetIO read splittable

2020-08-03 Thread GitBox
danielxjd commented on a change in pull request #12223: URL: https://github.com/apache/beam/pull/12223#discussion_r464701869 ## File path: sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java ## @@ -230,12 +281,271 @@ public ReadFiles

[GitHub] [beam] boyuanzz commented on a change in pull request #12419: [BEAM-10303] Handle split when truncate observes windows.

2020-08-03 Thread GitBox
boyuanzz commented on a change in pull request #12419: URL: https://github.com/apache/beam/pull/12419#discussion_r464700690 ## File path: sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java ## @@ -1035,6 +1046,130 @@ private Progress getProgress()

[GitHub] [beam] danielxjd commented on a change in pull request #12223: [Beam-4379] Make ParquetIO read splittable

2020-08-03 Thread GitBox
danielxjd commented on a change in pull request #12223: URL: https://github.com/apache/beam/pull/12223#discussion_r464700678 ## File path: sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java ## @@ -177,6 +214,10 @@ public Read from(String

[GitHub] [beam] leiyiz commented on pull request #12365: [BEAM-8258] python nexmark query 0-2 improvements

2020-08-03 Thread GitBox
leiyiz commented on pull request #12365: URL: https://github.com/apache/beam/pull/12365#issuecomment-668277274 > oops sorry what happened in this PR after all? I closed this draft and had opened a PR here: https://github.com/apache/beam/pull/12427

[GitHub] [beam] lukecwik commented on a change in pull request #12419: [BEAM-10303] Handle split when truncate observes windows.

2020-08-03 Thread GitBox
lukecwik commented on a change in pull request #12419: URL: https://github.com/apache/beam/pull/12419#discussion_r464699645 ## File path: sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java ## @@ -1193,6 +1328,7 @@ public Object restriction() {

[GitHub] [beam] lukecwik commented on a change in pull request #12419: [BEAM-10303] Handle split when truncate observes windows.

2020-08-03 Thread GitBox
lukecwik commented on a change in pull request #12419: URL: https://github.com/apache/beam/pull/12419#discussion_r464699501 ## File path: sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java ## @@ -1035,6 +1046,130 @@ private Progress getProgress()

[GitHub] [beam] lukecwik commented on a change in pull request #12419: [BEAM-10303] Handle split when truncate observes windows.

2020-08-03 Thread GitBox
lukecwik commented on a change in pull request #12419: URL: https://github.com/apache/beam/pull/12419#discussion_r464699157 ## File path: sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java ## @@ -1035,6 +1046,130 @@ private Progress getProgress()

[GitHub] [beam] danielxjd commented on pull request #12106: [BEAM-9792] Fixing the IOException handling in InsertAll for BigQuery

2020-08-03 Thread GitBox
danielxjd commented on pull request #12106: URL: https://github.com/apache/beam/pull/12106#issuecomment-668274752 @chamikaramj Do you have any additional comments? This is an automated message from the Apache Git Service. To

[GitHub] [beam] boyuanzz commented on a change in pull request #12419: [BEAM-10303] Handle split when truncate observes windows.

2020-08-03 Thread GitBox
boyuanzz commented on a change in pull request #12419: URL: https://github.com/apache/beam/pull/12419#discussion_r464697946 ## File path: sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java ## @@ -1035,6 +1046,130 @@ private Progress getProgress()

[GitHub] [beam] robertwb commented on a change in pull request #12352: [BEAM-10549] Improve runtime type checking performance for the Python SDK

2020-08-03 Thread GitBox
robertwb commented on a change in pull request #12352: URL: https://github.com/apache/beam/pull/12352#discussion_r464679106 ## File path: sdks/python/apache_beam/runners/worker/operations.py ## @@ -238,6 +247,7 @@ def __init__(self, self.execution_context = None # type:

[GitHub] [beam] boyuanzz commented on a change in pull request #12419: [BEAM-10303] Handle split when truncate observes windows.

2020-08-03 Thread GitBox
boyuanzz commented on a change in pull request #12419: URL: https://github.com/apache/beam/pull/12419#discussion_r464686872 ## File path: sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java ## @@ -1035,6 +1046,130 @@ private Progress getProgress()

[GitHub] [beam] lukecwik commented on a change in pull request #12430: [BEAM-10303] Scale progress with respect to windows observation.

2020-08-03 Thread GitBox
lukecwik commented on a change in pull request #12430: URL: https://github.com/apache/beam/pull/12430#discussion_r464673014 ## File path: sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java ## @@ -515,6 +515,9 @@ &&

[GitHub] [beam] apilloud commented on pull request #12416: Update google-api-services versions.

2020-08-03 Thread GitBox
apilloud commented on pull request #12416: URL: https://github.com/apache/beam/pull/12416#issuecomment-668260372 LGTM. There is a script `sdks/java/build-tools/beam-linkage-check.sh` that is suppose to validate version changes don't introduce new compatibility issues. Can you run it and

[GitHub] [beam] boyuanzz merged pull request #12371: [BEAM-10341] Refactor SplittableParDoExpander.

2020-08-03 Thread GitBox
boyuanzz merged pull request #12371: URL: https://github.com/apache/beam/pull/12371 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] chamikaramj commented on a change in pull request #12445: [BEAM-9919] Added an External Transform API to Go SDK

2020-08-03 Thread GitBox
chamikaramj commented on a change in pull request #12445: URL: https://github.com/apache/beam/pull/12445#discussion_r464672379 ## File path: runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/ValidateRunnerXlangTest.java ## @@ -177,6

[GitHub] [beam] aaltay commented on a change in pull request #12444: Added a whitespace lint as part of python lint precommit

2020-08-03 Thread GitBox
aaltay commented on a change in pull request #12444: URL: https://github.com/apache/beam/pull/12444#discussion_r464672903 ## File path: .test-infra/jenkins/job_PreCommit_Whitespace.groovy ## @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [beam] pabloem merged pull request #12331: [BEAM-10601] DICOM API Beam IO connector

2020-08-03 Thread GitBox
pabloem merged pull request #12331: URL: https://github.com/apache/beam/pull/12331 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] lukecwik commented on a change in pull request #12419: [BEAM-10303] Handle split when truncate observes windows.

2020-08-03 Thread GitBox
lukecwik commented on a change in pull request #12419: URL: https://github.com/apache/beam/pull/12419#discussion_r464651698 ## File path: sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java ## @@ -515,15 +515,18 @@ &&

[GitHub] [beam] boyuanzz commented on pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-08-03 Thread GitBox
boyuanzz commented on pull request #11749: URL: https://github.com/apache/beam/pull/11749#issuecomment-668251029 Based on the [readAll discussion](https://lists.apache.org/thread.html/rcb7538d5a2955d12e6d5f37a24830b929e5d65155d84f6d999801432%40%3Cdev.beam.apache.org%3E), I decided to go

[GitHub] [beam] robertwb commented on pull request #12459: [BEAM-9547] Simplify pandas implementation.

2020-08-03 Thread GitBox
robertwb commented on pull request #12459: URL: https://github.com/apache/beam/pull/12459#issuecomment-668251189 R: @apilloud This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] chamikaramj commented on a change in pull request #12223: [Beam-4379] Make ParquetIO read splittable

2020-08-03 Thread GitBox
chamikaramj commented on a change in pull request #12223: URL: https://github.com/apache/beam/pull/12223#discussion_r464659415 ## File path: sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java ## @@ -177,6 +214,10 @@ public Read from(String

[GitHub] [beam] boyuanzz commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-08-03 Thread GitBox
boyuanzz commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r464667015 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java ## @@ -906,19 +926,89 @@ public void

[GitHub] [beam] amaliujia edited a comment on pull request #12232: [Beam-9543] Support Match Recognition in Beam SQL

2020-08-03 Thread GitBox
amaliujia edited a comment on pull request #12232: URL: https://github.com/apache/beam/pull/12232#issuecomment-668147610 Interesting. I used the squash and merge button but it seems just create a merge commit. It might not matter that much.

[GitHub] [beam] pabloem commented on pull request #12442: Fixed pcoll visualization in datatable

2020-08-03 Thread GitBox
pabloem commented on pull request #12442: URL: https://github.com/apache/beam/pull/12442#issuecomment-668240989 thanks Ning! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] pabloem merged pull request #12442: Fixed pcoll visualization in datatable

2020-08-03 Thread GitBox
pabloem merged pull request #12442: URL: https://github.com/apache/beam/pull/12442 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] pabloem merged pull request #12339: Interactive: clean up when pipeline is out of scope

2020-08-03 Thread GitBox
pabloem merged pull request #12339: URL: https://github.com/apache/beam/pull/12339 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] KevinGG commented on a change in pull request #12339: Interactive: clean up when pipeline is out of scope

2020-08-03 Thread GitBox
KevinGG commented on a change in pull request #12339: URL: https://github.com/apache/beam/pull/12339#discussion_r464658479 ## File path: sdks/python/apache_beam/runners/interactive/interactive_environment.py ## @@ -364,26 +382,37 @@ def get_background_caching_job(self,

[GitHub] [beam] chamikaramj commented on pull request #12071: [BEAM-9932] Add documentation describing cross-language test pipelines

2020-08-03 Thread GitBox
chamikaramj commented on pull request #12071: URL: https://github.com/apache/beam/pull/12071#issuecomment-668237855 Seems like commit history got messed up. This is an automated message from the Apache Git Service. To

[GitHub] [beam] KevinGG commented on a change in pull request #12339: Interactive: clean up when pipeline is out of scope

2020-08-03 Thread GitBox
KevinGG commented on a change in pull request #12339: URL: https://github.com/apache/beam/pull/12339#discussion_r464657824 ## File path: sdks/python/apache_beam/runners/interactive/interactive_environment.py ## @@ -364,26 +382,37 @@ def get_background_caching_job(self,

[GitHub] [beam] robertwb opened a new pull request #12459: [BEAM-9547] Simplify pandas implementation.

2020-08-03 Thread GitBox
robertwb opened a new pull request #12459: URL: https://github.com/apache/beam/pull/12459 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose

[GitHub] [beam] pabloem commented on pull request #12331: [BEAM-10601] DICOM API Beam IO connector

2020-08-03 Thread GitBox
pabloem commented on pull request #12331: URL: https://github.com/apache/beam/pull/12331#issuecomment-668233131 LGTM. I'll merge once tests pass. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] pabloem commented on a change in pull request #12339: Interactive: clean up when pipeline is out of scope

2020-08-03 Thread GitBox
pabloem commented on a change in pull request #12339: URL: https://github.com/apache/beam/pull/12339#discussion_r464650111 ## File path: sdks/python/apache_beam/runners/interactive/interactive_environment.py ## @@ -364,26 +382,37 @@ def get_background_caching_job(self,

[GitHub] [beam] lostluck commented on a change in pull request #12426: [BEAM-7996] Add support for MapType and Nulls in container types for Python RowCoder

2020-08-03 Thread GitBox
lostluck commented on a change in pull request #12426: URL: https://github.com/apache/beam/pull/12426#discussion_r464647966 ## File path: sdks/python/apache_beam/coders/coder_impl.py ## @@ -530,6 +530,88 @@ def estimate_size(self, unused_value, nested=False): return 1

[GitHub] [beam] rosetn commented on pull request #12380: Cleanup WordCount example.

2020-08-03 Thread GitBox
rosetn commented on pull request #12380: URL: https://github.com/apache/beam/pull/12380#issuecomment-668225776 LGTM FYI @davidcavazos This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] pabloem commented on pull request #12365: [BEAM-8258] python nexmark query 0-2 improvements

2020-08-03 Thread GitBox
pabloem commented on pull request #12365: URL: https://github.com/apache/beam/pull/12365#issuecomment-668225000 oops sorry what happened in this PR after all? This is an automated message from the Apache Git Service. To

[GitHub] [beam] chamikaramj commented on pull request #12438: [BEAM-10618] subprocess_server.py: use AF_INET6 family when finding free port

2020-08-03 Thread GitBox
chamikaramj commented on pull request #12438: URL: https://github.com/apache/beam/pull/12438#issuecomment-668219156 Yeah, probably better to make INET6 a fallback. This is an automated message from the Apache Git Service. To

[GitHub] [beam] George-Wu commented on a change in pull request #12331: [BEAM-10601] DICOM API Beam IO connector

2020-08-03 Thread GitBox
George-Wu commented on a change in pull request #12331: URL: https://github.com/apache/beam/pull/12331#discussion_r464636772 ## File path: sdks/python/apache_beam/io/gcp/dicomio.py ## @@ -0,0 +1,572 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] [beam] amaliujia merged pull request #12436: [BEAM-9891] TPC-DS module initialization, tables and queries stored

2020-08-03 Thread GitBox
amaliujia merged pull request #12436: URL: https://github.com/apache/beam/pull/12436 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] robertwb commented on a change in pull request #12455: [BEAM-10630] Include data from load tests in the release process

2020-08-03 Thread GitBox
robertwb commented on a change in pull request #12455: URL: https://github.com/apache/beam/pull/12455#discussion_r464634785 ## File path: website/www/site/content/en/contribute/release-guide.md ## @@ -1163,7 +1180,7 @@ Use reporter.apache.org to seed the information about the

[GitHub] [beam] lostluck commented on a change in pull request #12350: [BEAM-10289] Dynamic splitting implementation.

2020-08-03 Thread GitBox
lostluck commented on a change in pull request #12350: URL: https://github.com/apache/beam/pull/12350#discussion_r464552611 ## File path: sdks/go/pkg/beam/core/runtime/exec/datasource_test.go ## @@ -18,6 +18,7 @@ package exec import ( "context" "fmt" +

[GitHub] [beam] robertwb merged pull request #12458: Fix dictionary changes size error in pickler.py

2020-08-03 Thread GitBox
robertwb merged pull request #12458: URL: https://github.com/apache/beam/pull/12458 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] robertwb merged pull request #12391: [BEAM-9839] OnTimerContext should not create a new one when processing each element/timer in FnApiDoFnRunner

2020-08-03 Thread GitBox
robertwb merged pull request #12391: URL: https://github.com/apache/beam/pull/12391 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] robertwb commented on pull request #12391: [BEAM-9839] OnTimerContext should not create a new one when processing each element/timer in FnApiDoFnRunner

2020-08-03 Thread GitBox
robertwb commented on pull request #12391: URL: https://github.com/apache/beam/pull/12391#issuecomment-668214822 The one failure looks like BEAM-10252. This is an automated message from the Apache Git Service. To respond to

[GitHub] [beam] George-Wu commented on a change in pull request #12331: [BEAM-10601] DICOM API Beam IO connector

2020-08-03 Thread GitBox
George-Wu commented on a change in pull request #12331: URL: https://github.com/apache/beam/pull/12331#discussion_r464632452 ## File path: sdks/python/apache_beam/io/gcp/dicomio.py ## @@ -0,0 +1,572 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] [beam] George-Wu commented on a change in pull request #12331: [BEAM-10601] DICOM API Beam IO connector

2020-08-03 Thread GitBox
George-Wu commented on a change in pull request #12331: URL: https://github.com/apache/beam/pull/12331#discussion_r464632452 ## File path: sdks/python/apache_beam/io/gcp/dicomio.py ## @@ -0,0 +1,572 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] [beam] robertwb commented on pull request #12391: [BEAM-9839] OnTimerContext should not create a new one when processing each element/timer in FnApiDoFnRunner

2020-08-03 Thread GitBox
robertwb commented on pull request #12391: URL: https://github.com/apache/beam/pull/12391#issuecomment-668213773 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.

2020-08-03 Thread GitBox
tysonjh commented on pull request #12416: URL: https://github.com/apache/beam/pull/12416#issuecomment-668212843 R: @apilloud Could you take a look at this please? This is an automated message from the Apache Git

  1   2   3   4   5   6   7   8   9   10   >