[GitHub] [beam] kamilwu commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
kamilwu commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435095271 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - re

[GitHub] [beam] kamilwu commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
kamilwu commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435099607 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - re

[GitHub] [beam] kamilwu commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
kamilwu commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435107715 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - re

[GitHub] [beam] mxm commented on pull request #11777: [BEAM-10054] Fix watermark hold for on_time_pane

2020-06-04 Thread GitBox
mxm commented on pull request #11777: URL: https://github.com/apache/beam/pull/11777#issuecomment-638750806 Since the test has been added and this was the missing piece here, I'm going to merge. This is an automated message

[GitHub] [beam] mxm merged pull request #11777: [BEAM-10054] Fix watermark hold for on_time_pane

2020-06-04 Thread GitBox
mxm merged pull request #11777: URL: https://github.com/apache/beam/pull/11777 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [beam] echauchot merged pull request #11732: [BEAM-10017] Expose Cassandra Connect and Read timeouts

2020-06-04 Thread GitBox
echauchot merged pull request #11732: URL: https://github.com/apache/beam/pull/11732 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] echauchot commented on pull request #11732: [BEAM-10017] Expose Cassandra Connect and Read timeouts

2020-06-04 Thread GitBox
echauchot commented on pull request #11732: URL: https://github.com/apache/beam/pull/11732#issuecomment-638758007 Did the final minor touches myself and merged the PR. Thanks Nathan for your work! This is an automated messag

[GitHub] [beam] iemejia commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests

2020-06-04 Thread GitBox
iemejia commented on pull request #11909: URL: https://github.com/apache/beam/pull/11909#issuecomment-638789937 Wow I am impressed they look REALLY nice great job! There are some things I would like to refine e.g. to be able to choose which runners to compare or just to choose one in

[GitHub] [beam] iemejia commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests

2020-06-04 Thread GitBox
iemejia commented on pull request #11909: URL: https://github.com/apache/beam/pull/11909#issuecomment-638790781 On extra nit pick: It would be nice if we could attribute a consistent color to each runner so in every view we can know the color corresponds to the runner. ---

[GitHub] [beam] TobKed commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
TobKed commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435190449 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - rel

[GitHub] [beam] TobKed commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
TobKed commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435192970 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - rel

[GitHub] [beam] TobKed commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
TobKed commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435203092 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - rel

[GitHub] [beam] TobKed commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
TobKed commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435203092 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - rel

[GitHub] [beam] TobKed commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
TobKed commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435212390 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - rel

[GitHub] [beam] kamilwu commented on pull request #11856: [BEAM-7505] SideInput Python Load Test job

2020-06-04 Thread GitBox
kamilwu commented on pull request #11856: URL: https://github.com/apache/beam/pull/11856#issuecomment-638816179 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [beam] kamilwu commented on pull request #11856: [BEAM-7505] SideInput Python Load Test job

2020-06-04 Thread GitBox
kamilwu commented on pull request #11856: URL: https://github.com/apache/beam/pull/11856#issuecomment-638825672 Run Python Load Tests SideInput Dataflow Batch This is an automated message from the Apache Git Service. To respo

[GitHub] [beam] damondouglas commented on a change in pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-04 Thread GitBox
damondouglas commented on a change in pull request #11883: URL: https://github.com/apache/beam/pull/11883#discussion_r435303784 ## File path: learning/katas/go/Core Transforms/Combine/CombineFn/pkg/task/task.go ## @@ -0,0 +1,55 @@ +// Licensed to the Apache Software Foundation

[GitHub] [beam] damondouglas commented on pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-04 Thread GitBox
damondouglas commented on pull request #11883: URL: https://github.com/apache/beam/pull/11883#issuecomment-63849 Thank you, @henryken for you recent comments. I have not yet updated stepik. If this meets your approval, I will update the course and commit the `*-remote.yaml` files to

[GitHub] [beam] kamilwu commented on a change in pull request #11856: [BEAM-7505] SideInput Python Load Test job

2020-06-04 Thread GitBox
kamilwu commented on a change in pull request #11856: URL: https://github.com/apache/beam/pull/11856#discussion_r435308264 ## File path: .test-infra/jenkins/job_LoadTests_SideInput_Python.groovy ## @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [beam] henryken commented on a change in pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-04 Thread GitBox
henryken commented on a change in pull request #11883: URL: https://github.com/apache/beam/pull/11883#discussion_r435314757 ## File path: learning/katas/go/Core Transforms/Combine/CombineFn/pkg/task/task.go ## @@ -0,0 +1,55 @@ +// Licensed to the Apache Software Foundation (ASF

[GitHub] [beam] henryken commented on a change in pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-04 Thread GitBox
henryken commented on a change in pull request #11883: URL: https://github.com/apache/beam/pull/11883#discussion_r435317881 ## File path: learning/katas/go/Core Transforms/Combine/CombineFn/pkg/task/task.go ## @@ -47,9 +47,9 @@ func (c *averageFn) MergeAccumulators(accumA, accu

[GitHub] [beam] dpcollins-google opened a new pull request #11919: [BEAM-10114] Copy Pub/Sub Lite IO from Pub/Sub Lite github to beam.

2020-06-04 Thread GitBox
dpcollins-google opened a new pull request #11919: URL: https://github.com/apache/beam/pull/11919 Also adapt to beam conventions and restrictions. This is staged here https://github.com/googleapis/java-pubsublite/tree/master/pubsublite-beam-io T

[GitHub] [beam] lostluck commented on a change in pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-04 Thread GitBox
lostluck commented on a change in pull request #11883: URL: https://github.com/apache/beam/pull/11883#discussion_r435344008 ## File path: learning/katas/go/Core Transforms/Combine/CombineFn/pkg/task/task.go ## @@ -0,0 +1,55 @@ +// Licensed to the Apache Software Foundation (ASF

[GitHub] [beam] damondouglas commented on a change in pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-04 Thread GitBox
damondouglas commented on a change in pull request #11883: URL: https://github.com/apache/beam/pull/11883#discussion_r435349866 ## File path: learning/katas/go/Core Transforms/Combine/CombineFn/pkg/task/task.go ## @@ -47,9 +47,9 @@ func (c *averageFn) MergeAccumulators(accumA,

[GitHub] [beam] lastomato opened a new pull request #11920: Minor formatting updates.

2020-06-04 Thread GitBox
lastomato opened a new pull request #11920: URL: https://github.com/apache/beam/pull/11920 - Chain constructors instead of assigning directly; - Typo fixes. The changes in this PR were from an internal code review. R: @pabloem cc @jaketf

[GitHub] [beam] lastomato commented on pull request #11920: Minor formatting updates.

2020-06-04 Thread GitBox
lastomato commented on pull request #11920: URL: https://github.com/apache/beam/pull/11920#issuecomment-638928233 I am not entirely sure if I need to create a JIRA issue for such a trivial change. This is an automated messag

[GitHub] [beam] aromanenko-dev commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-04 Thread GitBox
aromanenko-dev commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r435354750 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaViaSDF.java ## @@ -0,0 +1,697 @@ +/* + * Licensed to the A

[GitHub] [beam] aromanenko-dev commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-04 Thread GitBox
aromanenko-dev commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434729096 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaViaSDF.java ## @@ -0,0 +1,697 @@ +/* + * Licensed to the A

[GitHub] [beam] aromanenko-dev commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-04 Thread GitBox
aromanenko-dev commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r435359001 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java ## @@ -906,19 +955,110 @@ public void setValueDeserializer

[GitHub] [beam] chamikaramj commented on pull request #11919: [BEAM-10114] Copy Pub/Sub Lite IO from Pub/Sub Lite github to beam.

2020-06-04 Thread GitBox
chamikaramj commented on pull request #11919: URL: https://github.com/apache/beam/pull/11919#issuecomment-638938144 @dpmills @slavachernyak @reuvenlax are you interested in reviewing this ? This is an automated message from t

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11918: Add instructions to programmatically publish release to Github

2020-06-04 Thread GitBox
TheNeuralBit commented on a change in pull request #11918: URL: https://github.com/apache/beam/pull/11918#discussion_r435351070 ## File path: website/www/site/content/en/contribute/release-guide.md ## @@ -1134,12 +1134,29 @@ Create and push a new signed tag for the released ve

[GitHub] [beam] pabloem merged pull request #11898: [BEAM-10178] Improved error messages when option is not set

2020-06-04 Thread GitBox
pabloem merged pull request #11898: URL: https://github.com/apache/beam/pull/11898 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [beam] ibzib commented on a change in pull request #11918: Add instructions to programmatically publish release to Github

2020-06-04 Thread GitBox
ibzib commented on a change in pull request #11918: URL: https://github.com/apache/beam/pull/11918#discussion_r435356754 ## File path: website/www/site/content/en/contribute/release-guide.md ## @@ -1134,12 +1134,29 @@ Create and push a new signed tag for the released version b

[GitHub] [beam] ibzib commented on pull request #11918: Add instructions to programmatically publish release to Github

2020-06-04 Thread GitBox
ibzib commented on pull request #11918: URL: https://github.com/apache/beam/pull/11918#issuecomment-638943733 I didn't realize Brian had already reviewed... and we had almost the exact same comments 😆 This is an automated m

[GitHub] [beam] pabloem commented on pull request #11920: Minor formatting updates.

2020-06-04 Thread GitBox
pabloem commented on pull request #11920: URL: https://github.com/apache/beam/pull/11920#issuecomment-638947008 retest this please This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [beam] robertwb merged pull request #11787: [BEAM-10063] Better emulate the pandas testing environment.

2020-06-04 Thread GitBox
robertwb merged pull request #11787: URL: https://github.com/apache/beam/pull/11787 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] robertwb commented on a change in pull request #11787: [BEAM-10063] Better emulate the pandas testing environment.

2020-06-04 Thread GitBox
robertwb commented on a change in pull request #11787: URL: https://github.com/apache/beam/pull/11787#discussion_r435370735 ## File path: sdks/python/apache_beam/dataframe/doctests.py ## @@ -66,30 +93,19 @@ def __init__(self): self._all_frames = {} def fake_pandas_mod

[GitHub] [beam] y1chi commented on pull request #11916: [BEAM-10189] Add ValueState user state to python sdk

2020-06-04 Thread GitBox
y1chi commented on pull request #11916: URL: https://github.com/apache/beam/pull/11916#issuecomment-638950281 > Last time someone started adding this feature, we decided to call it ReadModifyWrite state. Java SDK still calls this ValueState, ReadModifyWriteState is only used in beam

[GitHub] [beam] robertwb commented on a change in pull request #11766: [BEAM-10036] More flexible dataframes partitioning.

2020-06-04 Thread GitBox
robertwb commented on a change in pull request #11766: URL: https://github.com/apache/beam/pull/11766#discussion_r435373732 ## File path: sdks/python/apache_beam/dataframe/expressions.py ## @@ -85,16 +87,10 @@ def evaluate_at(self, session): # type: (Session) -> T """Retu

[GitHub] [beam] robertwb commented on pull request #11835: Various fixes to allow Java PAssert to run on Python

2020-06-04 Thread GitBox
robertwb commented on pull request #11835: URL: https://github.com/apache/beam/pull/11835#issuecomment-638952489 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] henryken commented on a change in pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-04 Thread GitBox
henryken commented on a change in pull request #11883: URL: https://github.com/apache/beam/pull/11883#discussion_r435393888 ## File path: learning/katas/go/Core Transforms/Combine/CombineFn/pkg/task/task.go ## @@ -47,9 +47,9 @@ func (c *averageFn) MergeAccumulators(accumA, accu

[GitHub] [beam] lostluck commented on a change in pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-04 Thread GitBox
lostluck commented on a change in pull request #11883: URL: https://github.com/apache/beam/pull/11883#discussion_r435397313 ## File path: learning/katas/go/Core Transforms/Combine/CombineFn/pkg/task/task.go ## @@ -47,9 +47,9 @@ func (c *averageFn) MergeAccumulators(accumA, accu

[GitHub] [beam] ibzib opened a new pull request #11921: [BEAM-10194] Call println() in Katas test failures.

2020-06-04 Thread GitBox
ibzib opened a new pull request #11921: URL: https://github.com/apache/beam/pull/11921 R: @henryken Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s

[GitHub] [beam] henryken commented on pull request #11921: [BEAM-10194] Call println() in Katas test failures.

2020-06-04 Thread GitBox
henryken commented on pull request #11921: URL: https://github.com/apache/beam/pull/11921#issuecomment-638976587 @ibzib, may I know what kind of error that you encountered that needs this fix? This is an automated message fr

[GitHub] [beam] TobKed commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
TobKed commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435403856 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - rel

[GitHub] [beam] TobKed commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
TobKed commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435404202 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - rel

[GitHub] [beam] lukecwik merged pull request #11915: [BEAM-10190] Reduce cost of toString of MetricKey and MetricName

2020-06-04 Thread GitBox
lukecwik merged pull request #11915: URL: https://github.com/apache/beam/pull/11915 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] ibzib commented on pull request #11921: [BEAM-10194] Call println() in Katas test failures.

2020-06-04 Thread GitBox
ibzib commented on pull request #11921: URL: https://github.com/apache/beam/pull/11921#issuecomment-638977383 > @ibzib, may I know what kind of error that you encountered that needs this fix? Please see the related jira: https://issues.apache.org/jira/browse/BEAM-10194

[GitHub] [beam] lukecwik commented on pull request #11915: [BEAM-10190] Reduce cost of toString of MetricKey and MetricName

2020-06-04 Thread GitBox
lukecwik commented on pull request #11915: URL: https://github.com/apache/beam/pull/11915#issuecomment-638977891 Nice article on the performance impact of the different string concatenation methods. This is an automated mess

[GitHub] [beam] TobKed commented on pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
TobKed commented on pull request #11877: URL: https://github.com/apache/beam/pull/11877#issuecomment-638978380 > How can I preview the action on the fork: [TobKed#3](https://github.com/TobKed/beam/pull/3) ? @aaltay https://github.com/TobKed/beam/actions?query=branch%3Agithub-actions

[GitHub] [beam] TobKed commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-04 Thread GitBox
TobKed commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r435407993 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + - rel

[GitHub] [beam] aromanenko-dev commented on a change in pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write

2020-06-04 Thread GitBox
aromanenko-dev commented on a change in pull request #11396: URL: https://github.com/apache/beam/pull/11396#discussion_r435399974 ## File path: sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java ## @@ -1217,6 +1297,16 @@ void set( return toBuilder()

[GitHub] [beam] aromanenko-dev commented on pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write

2020-06-04 Thread GitBox
aromanenko-dev commented on pull request #11396: URL: https://github.com/apache/beam/pull/11396#issuecomment-638983908 retest this please This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [beam] aromanenko-dev commented on a change in pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write

2020-06-04 Thread GitBox
aromanenko-dev commented on a change in pull request #11396: URL: https://github.com/apache/beam/pull/11396#discussion_r435399974 ## File path: sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java ## @@ -1217,6 +1297,16 @@ void set( return toBuilder()

[GitHub] [beam] aromanenko-dev removed a comment on pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write

2020-06-04 Thread GitBox
aromanenko-dev removed a comment on pull request #11396: URL: https://github.com/apache/beam/pull/11396#issuecomment-638984073 retest this please This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [beam] aromanenko-dev removed a comment on pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write

2020-06-04 Thread GitBox
aromanenko-dev removed a comment on pull request #11396: URL: https://github.com/apache/beam/pull/11396#issuecomment-638983908 retest this please This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [beam] aromanenko-dev commented on pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write

2020-06-04 Thread GitBox
aromanenko-dev commented on pull request #11396: URL: https://github.com/apache/beam/pull/11396#issuecomment-638984333 retest this please This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [beam] aromanenko-dev commented on pull request #11396: [BEAM-9742] Add Configurable FluentBackoff to JdbcIO Write

2020-06-04 Thread GitBox
aromanenko-dev commented on pull request #11396: URL: https://github.com/apache/beam/pull/11396#issuecomment-638984073 retest this please This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [beam] chamikaramj commented on pull request #11846: [BEAM-9869] adding self-contained Kafka service jar for testing

2020-06-04 Thread GitBox
chamikaramj commented on pull request #11846: URL: https://github.com/apache/beam/pull/11846#issuecomment-638992107 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11766: [BEAM-10036] More flexible dataframes partitioning.

2020-06-04 Thread GitBox
TheNeuralBit commented on a change in pull request #11766: URL: https://github.com/apache/beam/pull/11766#discussion_r435422511 ## File path: sdks/python/apache_beam/dataframe/expressions.py ## @@ -85,16 +87,10 @@ def evaluate_at(self, session): # type: (Session) -> T """

[GitHub] [beam] damondouglas commented on pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-04 Thread GitBox
damondouglas commented on pull request #11883: URL: https://github.com/apache/beam/pull/11883#issuecomment-638992758 I've incorporated all the helpful comments. I'll wait for Henry's final approval before updating stepik/committing `*-remote.yaml` files. Thank you, both. --

[GitHub] [beam] xinyuiscool commented on pull request #11915: [BEAM-10190] Reduce cost of toString of MetricKey and MetricName

2020-06-04 Thread GitBox
xinyuiscool commented on pull request #11915: URL: https://github.com/apache/beam/pull/11915#issuecomment-639006539 @lukecwik : thanks for merging it! This is an automated message from the Apache Git Service. To respond to th

[GitHub] [beam] apilloud commented on pull request #11836: [BEAM-8693] Remove workaround for bigquery client

2020-06-04 Thread GitBox
apilloud commented on pull request #11836: URL: https://github.com/apache/beam/pull/11836#issuecomment-639007691 R: @suztomo This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] Zhangyx39 commented on pull request #11915: [BEAM-10190] Reduce cost of toString of MetricKey and MetricName

2020-06-04 Thread GitBox
Zhangyx39 commented on pull request #11915: URL: https://github.com/apache/beam/pull/11915#issuecomment-639009096 Thank you, Likasz and Xinyu. This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [beam] suztomo commented on pull request #11836: [BEAM-8693] Remove workaround for bigquery client

2020-06-04 Thread GitBox
suztomo commented on pull request #11836: URL: https://github.com/apache/beam/pull/11836#issuecomment-639010785 Does `sdks/java/build-tools/beam-linkage-check.sh` ([wiki](https://cwiki.apache.org/confluence/display/BEAM/Dependency+Upgrades)), without arguments, complain anything now? If

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11766: [BEAM-10036] More flexible dataframes partitioning.

2020-06-04 Thread GitBox
TheNeuralBit commented on a change in pull request #11766: URL: https://github.com/apache/beam/pull/11766#discussion_r435426414 ## File path: sdks/python/apache_beam/dataframe/partitionings.py ## @@ -0,0 +1,136 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [beam] lukecwik commented on pull request #11922: [BEAM-2939] Fix FnApiDoFnRunner to ensure that we output within the correct window.

2020-06-04 Thread GitBox
lukecwik commented on pull request #11922: URL: https://github.com/apache/beam/pull/11922#issuecomment-639020928 R: @boyuanzz @chamikaramj This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] lukecwik opened a new pull request #11922: [BEAM-2939] Fix FnApiDoFnRunner to ensure that we output within the correct window.

2020-06-04 Thread GitBox
lukecwik opened a new pull request #11922: URL: https://github.com/apache/beam/pull/11922 This fixes a bug where we would output within all the windows instead of just the current window. This would not impact any SDF that used only a single window while processing. -

[GitHub] [beam] apilloud commented on pull request #11836: [BEAM-8693] Remove workaround for bigquery client

2020-06-04 Thread GitBox
apilloud commented on pull request #11836: URL: https://github.com/apache/beam/pull/11836#issuecomment-639023595 I get `No new linkage errors`. This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [beam] apilloud merged pull request #11836: [BEAM-8693] Remove workaround for bigquery client

2020-06-04 Thread GitBox
apilloud merged pull request #11836: URL: https://github.com/apache/beam/pull/11836 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] chunyang opened a new pull request #11923: [BEAM-10176] Support STRUCT, FLOAT64, INT64 BigQuery types

2020-06-04 Thread GitBox
chunyang opened a new pull request #11923: URL: https://github.com/apache/beam/pull/11923 Support STRUCT, FLOAT64, INT64 BigQuery types in BIgQuery to Avro schema conversion. We already support RECORD, FLOAT, and INTEGER, which are aliases of the three types being added.

[GitHub] [beam] chunyang commented on pull request #11923: [BEAM-10176] Support STRUCT, FLOAT64, INT64 BigQuery types

2020-06-04 Thread GitBox
chunyang commented on pull request #11923: URL: https://github.com/apache/beam/pull/11923#issuecomment-639047436 R: @pabloem This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] jaketf commented on pull request #11893: [BEAM-10175] use bearer token header instead of url param for token

2020-06-04 Thread GitBox
jaketf commented on pull request #11893: URL: https://github.com/apache/beam/pull/11893#issuecomment-639055068 Hmmm my latest commit was just `spotlessApply` and this entire PR's change (how we pass the auth token for HCAPI requests for FHIR) seems unrelated to the GCS object moving logic

[GitHub] [beam] jaketf edited a comment on pull request #11893: [BEAM-10175] use bearer token header instead of url param for token

2020-06-04 Thread GitBox
jaketf edited a comment on pull request #11893: URL: https://github.com/apache/beam/pull/11893#issuecomment-639055068 Hmmm my latest commit was just `spotlessApply` and this entire PR's change (how we pass the auth token for HCAPI requests for FHIR) seems unrelated to the GCS object moving

[GitHub] [beam] jphalip commented on pull request #11918: [BEAM-10188] Add instructions to programmatically publish release to Github

2020-06-04 Thread GitBox
jphalip commented on pull request #11918: URL: https://github.com/apache/beam/pull/11918#issuecomment-639058954 @ibzib @TheNeuralBit Thank you both for the feedback. I've moved the code to a separate script. All the variables used should already be set in `script.config`. Does that work?

[GitHub] [beam] pabloem commented on pull request #11893: [BEAM-10175] use bearer token header instead of url param for token

2020-06-04 Thread GitBox
pabloem commented on pull request #11893: URL: https://github.com/apache/beam/pull/11893#issuecomment-639059270 Ah great observation! I'll merge. I am not aware of it being flaky otherwise. This is an automated message from t

[GitHub] [beam] pabloem merged pull request #11893: [BEAM-10175] use bearer token header instead of url param for token

2020-06-04 Thread GitBox
pabloem merged pull request #11893: URL: https://github.com/apache/beam/pull/11893 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [beam] bashir2 commented on pull request #11617: Update programming-guide.md with some typo fixes.

2020-06-04 Thread GitBox
bashir2 commented on pull request #11617: URL: https://github.com/apache/beam/pull/11617#issuecomment-639060524 Closing this for now; will try to recreate another one later if I come back to Beam documentation. This is an au

[GitHub] [beam] bashir2 closed pull request #11617: Update programming-guide.md with some typo fixes.

2020-06-04 Thread GitBox
bashir2 closed pull request #11617: URL: https://github.com/apache/beam/pull/11617 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [beam] ibzib commented on pull request #11918: [BEAM-10188] Add instructions to programmatically publish release to Github

2020-06-04 Thread GitBox
ibzib commented on pull request #11918: URL: https://github.com/apache/beam/pull/11918#issuecomment-639062174 > All the variables used should already be set in script.config. Does that work? That's probably fine, but if that's the expectation we need to `source script.config`. ---

[GitHub] [beam] jphalip commented on pull request #11918: [BEAM-10188] Add instructions to programmatically publish release to Github

2020-06-04 Thread GitBox
jphalip commented on pull request #11918: URL: https://github.com/apache/beam/pull/11918#issuecomment-639064988 Ok, I've loaded `script.config`. I've also added a confirmation message before submitting the API request. This

[GitHub] [beam] pabloem commented on pull request #11920: Minor formatting updates.

2020-06-04 Thread GitBox
pabloem commented on pull request #11920: URL: https://github.com/apache/beam/pull/11920#issuecomment-639065558 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [beam] rehmanmuradali opened a new pull request #11924: [BEAM-8543] Dataflow streaming timers are not strictly time ordered when set earlier mid-bundle

2020-06-04 Thread GitBox
rehmanmuradali opened a new pull request #11924: URL: https://github.com/apache/beam/pull/11924 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**]

[GitHub] [beam] rehmanmuradali commented on pull request #11924: [BEAM-8543] Dataflow streaming timers are not strictly time ordered when set earlier mid-bundle

2020-06-04 Thread GitBox
rehmanmuradali commented on pull request #11924: URL: https://github.com/apache/beam/pull/11924#issuecomment-639088647 R: @reuvenlax Could you please take a look that I am on right track? This is an automated message fro

[GitHub] [beam] vmarquez commented on a change in pull request #10546: [BEAM-9008] Add CassandraIO readAll method

2020-06-04 Thread GitBox
vmarquez commented on a change in pull request #10546: URL: https://github.com/apache/beam/pull/10546#discussion_r434870508 ## File path: sdks/java/io/cassandra/src/main/java/org/apache/beam/sdk/io/cassandra/CassandraIO.java ## @@ -1170,4 +898,44 @@ private void waitForFutures

[GitHub] [beam] boyuanzz commented on a change in pull request #11922: [BEAM-2939] Fix FnApiDoFnRunner to ensure that we output within the correct window when processing a splittable dofn

2020-06-04 Thread GitBox
boyuanzz commented on a change in pull request #11922: URL: https://github.com/apache/beam/pull/11922#discussion_r435521657 ## File path: sdks/java/harness/src/test/java/org/apache/beam/fn/harness/FnApiDoFnRunnerTest.java ## @@ -1593,6 +1601,391 @@ public void testProcessElem

[GitHub] [beam] TheNeuralBit merged pull request #11918: [BEAM-10188] Add instructions to programmatically publish release to Github

2020-06-04 Thread GitBox
TheNeuralBit merged pull request #11918: URL: https://github.com/apache/beam/pull/11918 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] chamikaramj commented on pull request #11846: [BEAM-9869] adding self-contained Kafka service jar for testing

2020-06-04 Thread GitBox
chamikaramj commented on pull request #11846: URL: https://github.com/apache/beam/pull/11846#issuecomment-639098591 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [beam] pabloem merged pull request #11920: Minor formatting updates.

2020-06-04 Thread GitBox
pabloem merged pull request #11920: URL: https://github.com/apache/beam/pull/11920 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [beam] lukecwik commented on pull request #11821: [BEAM-10097, BEAM-5982, BEAM-3080] Use primitive views directly instead of transforming KV> to the view type via a naive ma

2020-06-04 Thread GitBox
lukecwik commented on pull request #11821: URL: https://github.com/apache/beam/pull/11821#issuecomment-639108631 R: @Ardagan @amaliujia This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [beam] apilloud commented on a change in pull request #11868: [BEAM-9363] Support HOP and SESSION as TVF

2020-06-04 Thread GitBox
apilloud commented on a change in pull request #11868: URL: https://github.com/apache/beam/pull/11868#discussion_r435497089 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/TVFSlidingWindowFn.java ## @@ -0,0 +1,101 @@ +/* + * License

[GitHub] [beam] lukecwik commented on a change in pull request #11922: [BEAM-2939] Fix FnApiDoFnRunner to ensure that we output within the correct window when processing a splittable dofn

2020-06-04 Thread GitBox
lukecwik commented on a change in pull request #11922: URL: https://github.com/apache/beam/pull/11922#discussion_r435546832 ## File path: sdks/java/harness/src/test/java/org/apache/beam/fn/harness/FnApiDoFnRunnerTest.java ## @@ -1593,6 +1601,391 @@ public void testProcessElem

[GitHub] [beam] lostluck opened a new pull request #11925: [BEAM-9615] Add string coder utility functions.

2020-06-04 Thread GitBox
lostluck opened a new pull request #11925: URL: https://github.com/apache/beam/pull/11925 This adds initial utility functions for encoding and decoding utf8 strings in the Go SDK. Doesn't make use of them yet. In practice this is already how strings are encoded in the Go SDK, but ma

[GitHub] [beam] lostluck commented on pull request #11925: [BEAM-9615] Add string coder utility functions.

2020-06-04 Thread GitBox
lostluck commented on pull request #11925: URL: https://github.com/apache/beam/pull/11925#issuecomment-639119430 R: @tysonjh cc: @youngoli This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [beam] lostluck commented on pull request #11926: [BEAM-9615] Additional coder unit tests

2020-06-04 Thread GitBox
lostluck commented on pull request #11926: URL: https://github.com/apache/beam/pull/11926#issuecomment-639122665 R: @tysonjh cc: @youngoli This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [beam] lostluck opened a new pull request #11926: [BEAM-9615] Additional coder unit tests

2020-06-04 Thread GitBox
lostluck opened a new pull request #11926: URL: https://github.com/apache/beam/pull/11926 Adds a unit test for the "user side" type encoders /decoders. This will help avoiding having a broken state as we change the string coder, and default custom type coders. Also adjusts the coder

[GitHub] [beam] robertwb commented on pull request #11835: Various fixes to allow Java PAssert to run on Python

2020-06-04 Thread GitBox
robertwb commented on pull request #11835: URL: https://github.com/apache/beam/pull/11835#issuecomment-639152041 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [beam] robertwb commented on a change in pull request #11894: [BEAM-7074] Enable test_pardo_timers_clear for fn_runner

2020-06-04 Thread GitBox
robertwb commented on a change in pull request #11894: URL: https://github.com/apache/beam/pull/11894#discussion_r435595283 ## File path: sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner_test.py ## @@ -377,11 +377,6 @@ def process_timer( assert_that(a

[GitHub] [beam] kennknowles commented on a change in pull request #11820: [BEAM-10093] ZetaSql Nexmark variant

2020-06-04 Thread GitBox
kennknowles commented on a change in pull request #11820: URL: https://github.com/apache/beam/pull/11820#discussion_r435597049 ## File path: sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/zetasql/ZetaSqlQuery0.java ## @@ -0,0 +1,92 @@ +/* + * Licen

[GitHub] [beam] kennknowles commented on a change in pull request #11820: [BEAM-10093] ZetaSql Nexmark variant

2020-06-04 Thread GitBox
kennknowles commented on a change in pull request #11820: URL: https://github.com/apache/beam/pull/11820#discussion_r435597049 ## File path: sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/zetasql/ZetaSqlQuery0.java ## @@ -0,0 +1,92 @@ +/* + * Licen

  1   2   >