[GitHub] [beam] henryken commented on a change in pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-03 Thread GitBox
henryken commented on a change in pull request #11883: URL: https://github.com/apache/beam/pull/11883#discussion_r434997866 ## File path: learning/katas/go/Core Transforms/Combine/CombineFn/pkg/task/task.go ## @@ -0,0 +1,55 @@ +// Licensed to the Apache Software Foundation

[GitHub] [beam] robertwb commented on pull request #11835: Various fixes to allow Java PAssert to run on Python

2020-06-03 Thread GitBox
robertwb commented on pull request #11835: URL: https://github.com/apache/beam/pull/11835#issuecomment-638591806 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] robertwb commented on pull request #11835: Various fixes to allow Java PAssert to run on Python

2020-06-03 Thread GitBox
robertwb commented on pull request #11835: URL: https://github.com/apache/beam/pull/11835#issuecomment-638591762 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] lukecwik merged pull request #11917: [BEAM-8602] Use shadow configuration for remaining DirectRunner dependencies

2020-06-03 Thread GitBox
lukecwik merged pull request #11917: URL: https://github.com/apache/beam/pull/11917 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] chamikaramj commented on a change in pull request #11495: [BEAM-9802] Customization of automatically started services.

2020-06-03 Thread GitBox
chamikaramj commented on a change in pull request #11495: URL: https://github.com/apache/beam/pull/11495#discussion_r434945430 ## File path: sdks/python/apache_beam/options/pipeline_options.py ## @@ -425,6 +425,24 @@ def _add_argparse_args(cls, parser): help='Whether

[GitHub] [beam] pabloem commented on pull request #11898: [BEAM-10178] Improved error messages when option is not set

2020-06-03 Thread GitBox
pabloem commented on pull request #11898: URL: https://github.com/apache/beam/pull/11898#issuecomment-638545151 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] robertwb commented on pull request #11916: [BEAM-10189] Add ValueState user state to python sdk

2020-06-03 Thread GitBox
robertwb commented on pull request #11916: URL: https://github.com/apache/beam/pull/11916#issuecomment-638541628 Last time someone started adding this feature, we decided to call it ReadModifyWrite state. This is an

[GitHub] [beam] TheNeuralBit edited a comment on pull request #11917: [BEAM-8602] Use shadow configuration for remaining DirectRunner dependencies

2020-06-03 Thread GitBox
TheNeuralBit edited a comment on pull request #11917: URL: https://github.com/apache/beam/pull/11917#issuecomment-638530705 I grepped the build.gradle files to look for any other instances of adding the direct runner as a dependency dynamically, and couldn't find anything else: ```

[GitHub] [beam] TheNeuralBit commented on pull request #11917: [BEAM-8602] Use shadow configuration for remaining DirectRunner dependencies

2020-06-03 Thread GitBox
TheNeuralBit commented on pull request #11917: URL: https://github.com/apache/beam/pull/11917#issuecomment-638530705 I grepped the build.gradle files to look for any other instances of adding the direct runner as a dependency dynamically, and couldn't find anything else: ``` ❯

[GitHub] [beam] TheNeuralBit opened a new pull request #11917: [BEAM-8602] Use shadow configuration for remaining DirectRunner dependencies

2020-06-03 Thread GitBox
TheNeuralBit opened a new pull request #11917: URL: https://github.com/apache/beam/pull/11917 R: @lukecwik Post-Commit Tests Status (on master branch) Lang | SDK | Apex |

[GitHub] [beam] y1chi opened a new pull request #11916: [BEAM-10189] Add ValueState user state to python sdk

2020-06-03 Thread GitBox
y1chi opened a new pull request #11916: URL: https://github.com/apache/beam/pull/11916 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly

[GitHub] [beam] y1chi commented on pull request #11916: [BEAM-10189] Add ValueState user state to python sdk

2020-06-03 Thread GitBox
y1chi commented on pull request #11916: URL: https://github.com/apache/beam/pull/11916#issuecomment-638528628 R: @angoenka @robertwb This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] amaliujia merged pull request #11895: [Beam 9879] support STRING_AGG function in Beam ZetaSQL

2020-06-03 Thread GitBox
amaliujia merged pull request #11895: URL: https://github.com/apache/beam/pull/11895 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11787: [BEAM-10063] Better emulate the pandas testing environment.

2020-06-03 Thread GitBox
TheNeuralBit commented on a change in pull request #11787: URL: https://github.com/apache/beam/pull/11787#discussion_r434919303 ## File path: sdks/python/apache_beam/dataframe/doctests.py ## @@ -66,30 +93,19 @@ def __init__(self): self._all_frames = {} def

[GitHub] [beam] xinyuiscool commented on pull request #11915: [BEAM-10190] Reduce cost of toString of MetricKey and MetricName

2020-06-03 Thread GitBox
xinyuiscool commented on pull request #11915: URL: https://github.com/apache/beam/pull/11915#issuecomment-638522514 Sure, I will take a look. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11766: [BEAM-10036] More flexible dataframes partitioning.

2020-06-03 Thread GitBox
TheNeuralBit commented on a change in pull request #11766: URL: https://github.com/apache/beam/pull/11766#discussion_r434916844 ## File path: sdks/python/apache_beam/dataframe/expressions.py ## @@ -85,16 +87,10 @@ def evaluate_at(self, session): # type: (Session) -> T

[GitHub] [beam] chamikaramj commented on pull request #11846: [BEAM-9869] adding self-contained Kafka service jar for testing

2020-06-03 Thread GitBox
chamikaramj commented on pull request #11846: URL: https://github.com/apache/beam/pull/11846#issuecomment-638516729 Retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] amaliujia commented on pull request #11895: [Beam 9879] support STRING_AGG function in Beam ZetaSQL

2020-06-03 Thread GitBox
amaliujia commented on pull request #11895: URL: https://github.com/apache/beam/pull/11895#issuecomment-638514897 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] ibzib merged pull request #11914: [BEAM-10187] Push release candidate tag to Github.

2020-06-03 Thread GitBox
ibzib merged pull request #11914: URL: https://github.com/apache/beam/pull/11914 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] Zhangyx39 commented on pull request #11915: [BEAM-10190] Reduce cost of toString of MetricKey and MetricName

2020-06-03 Thread GitBox
Zhangyx39 commented on pull request #11915: URL: https://github.com/apache/beam/pull/11915#issuecomment-638497048 @xinyuiscool would you like to take a look at this pr? This is an automated message from the Apache Git

[GitHub] [beam] Zhangyx39 opened a new pull request #11915: [BEAM-10190] Reduce cost of toString of MetricKey and MetricName

2020-06-03 Thread GitBox
Zhangyx39 opened a new pull request #11915: URL: https://github.com/apache/beam/pull/11915 Samza runner heavily uses MetricKey.toString() and MetricName.toString() to update Samza metrics. We found that the toString methods have high CPU cost. And according to this article: 

[GitHub] [beam] boyuanzz commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
boyuanzz commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434890835 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java ## @@ -817,6 +847,24 @@ public void setValueDeserializer(String

[GitHub] [beam] damondouglas commented on pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-03 Thread GitBox
damondouglas commented on pull request #11883: URL: https://github.com/apache/beam/pull/11883#issuecomment-638488229 @lostluck I was waiting on @henryken approval. Should I just go ahead and update stepik and commit the `*-remote.yaml` to this PR?

[GitHub] [beam] vmarquez commented on a change in pull request #10546: [BEAM-9008] Add CassandraIO readAll method

2020-06-03 Thread GitBox
vmarquez commented on a change in pull request #10546: URL: https://github.com/apache/beam/pull/10546#discussion_r434870508 ## File path: sdks/java/io/cassandra/src/main/java/org/apache/beam/sdk/io/cassandra/CassandraIO.java ## @@ -1170,4 +898,44 @@ private void

[GitHub] [beam] ibzib commented on a change in pull request #11739: [BEAM-10024] Mark testOutputTimestampDefault with UsesUnboundedPColle…

2020-06-03 Thread GitBox
ibzib commented on a change in pull request #11739: URL: https://github.com/apache/beam/pull/11739#discussion_r434869939 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoTest.java ## @@ -3539,6 +3539,7 @@ public void onTimer() {} @Category({

[GitHub] [beam] lostluck commented on pull request #11883: [BEAM-9679] Add CombineFn to Core Transforms

2020-06-03 Thread GitBox
lostluck commented on pull request #11883: URL: https://github.com/apache/beam/pull/11883#issuecomment-638471588 @damondouglas Has stepik been updated? Is this clear to merge? This is an automated message from the Apache Git

[GitHub] [beam] lostluck merged pull request #11881: [BEAM-10166] Context improvements for exec errors.

2020-06-03 Thread GitBox
lostluck merged pull request #11881: URL: https://github.com/apache/beam/pull/11881 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] ibzib commented on a change in pull request #11914: [BEAM-10187] Push release candidate tag to Github.

2020-06-03 Thread GitBox
ibzib commented on a change in pull request #11914: URL: https://github.com/apache/beam/pull/11914#discussion_r434850943 ## File path: release/src/main/scripts/build_release_candidate.sh ## @@ -106,9 +106,7 @@ if [[ $confirmation = "y" ]]; then

[GitHub] [beam] KevinGG commented on pull request #11898: [BEAM-10178] Improved error messages when option is not set

2020-06-03 Thread GitBox
KevinGG commented on pull request #11898: URL: https://github.com/apache/beam/pull/11898#issuecomment-638458752 > Run Java PreCommit Thanks Pablo! This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] pabloem commented on pull request #11898: [BEAM-10178] Improved error messages when option is not set

2020-06-03 Thread GitBox
pabloem commented on pull request #11898: URL: https://github.com/apache/beam/pull/11898#issuecomment-638457890 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] pabloem commented on pull request #11893: [BEAM-10175] use bearer token header instead of url param for token

2020-06-03 Thread GitBox
pabloem commented on pull request #11893: URL: https://github.com/apache/beam/pull/11893#issuecomment-638458169 I'm ready to merge. I jsut want to confirm whether the Fhir import test is flaky, or if you've fixed it in your last commit? @jaketf

[GitHub] [beam] pabloem merged pull request #11896: [BEAM-8841] Make bigquery_avro_tools_test py3 compatible

2020-06-03 Thread GitBox
pabloem merged pull request #11896: URL: https://github.com/apache/beam/pull/11896 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11914: [BEAM-10187] Push release candidate tag to Github.

2020-06-03 Thread GitBox
TheNeuralBit commented on a change in pull request #11914: URL: https://github.com/apache/beam/pull/11914#discussion_r434847753 ## File path: release/src/main/scripts/build_release_candidate.sh ## @@ -106,9 +106,7 @@ if [[ $confirmation = "y" ]]; then

[GitHub] [beam] ibzib commented on a change in pull request #11914: [BEAM-10187] Push release candidate tag to Github.

2020-06-03 Thread GitBox
ibzib commented on a change in pull request #11914: URL: https://github.com/apache/beam/pull/11914#discussion_r434844066 ## File path: release/src/main/scripts/build_release_candidate.sh ## @@ -106,9 +106,7 @@ if [[ $confirmation = "y" ]]; then

[GitHub] [beam] chamikaramj commented on pull request #11846: [BEAM-9869] adding self-contained Kafka service jar for testing

2020-06-03 Thread GitBox
chamikaramj commented on pull request #11846: URL: https://github.com/apache/beam/pull/11846#issuecomment-638451041 Retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11914: [BEAM-10187] Push release candidate tag to Github.

2020-06-03 Thread GitBox
TheNeuralBit commented on a change in pull request #11914: URL: https://github.com/apache/beam/pull/11914#discussion_r434840793 ## File path: release/src/main/scripts/build_release_candidate.sh ## @@ -106,9 +106,7 @@ if [[ $confirmation = "y" ]]; then

[GitHub] [beam] ibzib opened a new pull request #11914: [BEAM-10187] Push release candidate tag to Github.

2020-06-03 Thread GitBox
ibzib opened a new pull request #11914: URL: https://github.com/apache/beam/pull/11914 R: @TheNeuralBit @amaliujia Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ]

[GitHub] [beam] boyuanzz commented on pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
boyuanzz commented on pull request #11749: URL: https://github.com/apache/beam/pull/11749#issuecomment-638439090 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] robertwb commented on pull request #11912: [BEAM-10165] Cache and return error messages on pipeline failure.

2020-06-03 Thread GitBox
robertwb commented on pull request #11912: URL: https://github.com/apache/beam/pull/11912#issuecomment-638439054 R: @angoenka This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] robertwb commented on pull request #11913: Cleaner logging, explicit errors on unimplemented features.

2020-06-03 Thread GitBox
robertwb commented on pull request #11913: URL: https://github.com/apache/beam/pull/11913#issuecomment-638438893 R: @kennknowles This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] robertwb opened a new pull request #11913: Cleaner logging, explicit errors on unimplemented features.

2020-06-03 Thread GitBox
robertwb opened a new pull request #11913: URL: https://github.com/apache/beam/pull/11913 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose

[GitHub] [beam] robertwb opened a new pull request #11912: [BEAM-10165] Cache and return error messages on pipeline failure.

2020-06-03 Thread GitBox
robertwb opened a new pull request #11912: URL: https://github.com/apache/beam/pull/11912 The Java portable runner only fetches messages and looks for errors after the pipeline returns failure. Thank you for your contribution! Follow this checklist

[GitHub] [beam] chamikaramj commented on pull request #11911: [BEAM-10186] Send an error to the runner for certain split and progress failures

2020-06-03 Thread GitBox
chamikaramj commented on pull request #11911: URL: https://github.com/apache/beam/pull/11911#issuecomment-638427486 Retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] chamikaramj commented on pull request #11911: [BEAM-10186] Send an error to the runner for certain split and progress failures

2020-06-03 Thread GitBox
chamikaramj commented on pull request #11911: URL: https://github.com/apache/beam/pull/11911#issuecomment-638427096 R: @lukecwik This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] chamikaramj opened a new pull request #11911: [BEAM-10186] Send an error to the runner for certain split and progress failures

2020-06-03 Thread GitBox
chamikaramj opened a new pull request #11911: URL: https://github.com/apache/beam/pull/11911 Send an error to the runner for certain split and progress request failures. This is needed to prevent runner from waiting unnecessarily and timing out for progress and split requests that

[GitHub] [beam] amaliujia commented on pull request #11895: [Beam 9879] support STRING_AGG function in Beam ZetaSQL

2020-06-03 Thread GitBox
amaliujia commented on pull request #11895: URL: https://github.com/apache/beam/pull/11895#issuecomment-638424051 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] pabloem commented on pull request #11896: [BEAM-8841] Make bigquery_avro_tools_test py3 compatible

2020-06-03 Thread GitBox
pabloem commented on pull request #11896: URL: https://github.com/apache/beam/pull/11896#issuecomment-638416678 Run PythonLint PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11904: [DO NOT MERGE] Update Beam website to release 2.22.0

2020-06-03 Thread GitBox
TheNeuralBit commented on a change in pull request #11904: URL: https://github.com/apache/beam/pull/11904#discussion_r434800889 ## File path: website/www/site/content/en/get-started/downloads.md ## @@ -87,6 +87,13 @@ versions denoted `0.x.y`. ## Releases +### 2.22.0

[GitHub] [beam] TheNeuralBit merged pull request #11903: Fix hub existence test in RC script

2020-06-03 Thread GitBox
TheNeuralBit merged pull request #11903: URL: https://github.com/apache/beam/pull/11903 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [beam] TheNeuralBit commented on pull request #11903: Fix hub existence test in RC script

2020-06-03 Thread GitBox
TheNeuralBit commented on pull request #11903: URL: https://github.com/apache/beam/pull/11903#issuecomment-638410392 CI failure is unrelated. I'll go ahead and merge. This is an automated message from the Apache Git Service.

[GitHub] [beam] pabloem commented on pull request #11896: [BEAM-8841] Make bigquery_avro_tools_test py3 compatible

2020-06-03 Thread GitBox
pabloem commented on pull request #11896: URL: https://github.com/apache/beam/pull/11896#issuecomment-638403555 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] pabloem commented on pull request #11896: [BEAM-8841] Make bigquery_avro_tools_test py3 compatible

2020-06-03 Thread GitBox
pabloem commented on pull request #11896: URL: https://github.com/apache/beam/pull/11896#issuecomment-638403708 thanks @chunyang ! this LGTM. I'll merge after lint passes This is an automated message from the Apache Git

[GitHub] [beam] TheNeuralBit merged pull request #11900: [BEAM-10181] Always use python3 for license script virtualenv

2020-06-03 Thread GitBox
TheNeuralBit merged pull request #11900: URL: https://github.com/apache/beam/pull/11900 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11910: Add blog post announcing the 2.22.0 release

2020-06-03 Thread GitBox
TheNeuralBit commented on a change in pull request #11910: URL: https://github.com/apache/beam/pull/11910#discussion_r434786163 ## File path: website/www/site/content/en/blog/beam-2.22.0.md ## @@ -0,0 +1,53 @@ +--- +title: "Apache Beam 2.21.0" +date: 2020-05-27 00:00:01

[GitHub] [beam] TheNeuralBit opened a new pull request #11910: Add blog post announcing the 2.22.0 release

2020-06-03 Thread GitBox
TheNeuralBit opened a new pull request #11910: URL: https://github.com/apache/beam/pull/11910 Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink |

[GitHub] [beam] aaltay commented on pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-03 Thread GitBox
aaltay commented on pull request #11877: URL: https://github.com/apache/beam/pull/11877#issuecomment-638395828 Could we make it such that: - we have a cron job that builds nightly (like a snapshot release) - and a way to manually trigger this from the release branch, so that the

[GitHub] [beam] tvalentyn commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-03 Thread GitBox
tvalentyn commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r434779681 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + -

[GitHub] [beam] TheNeuralBit commented on a change in pull request #11901: Prototype schema-inferring Row constructor.

2020-06-03 Thread GitBox
TheNeuralBit commented on a change in pull request #11901: URL: https://github.com/apache/beam/pull/11901#discussion_r434772179 ## File path: sdks/python/apache_beam/transforms/sql.py ## @@ -74,3 +77,8 @@ def __init__(self, query, dialect=None):

[GitHub] [beam] thinhha commented on pull request #11873: [BEAM-3489] DataflowRunner to read PubsubMessage if getNeedsMessageId is true

2020-06-03 Thread GitBox
thinhha commented on pull request #11873: URL: https://github.com/apache/beam/pull/11873#issuecomment-638380598 Thanks @lukecwik. I've updated the change. I thought I had changed it back to your suggestion before!

[GitHub] [beam] boyuanzz commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
boyuanzz commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434766297 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaViaSDF.java ## @@ -0,0 +1,697 @@ +/* + * Licensed to the Apache

[GitHub] [beam] boyuanzz commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
boyuanzz commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434746269 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIOUtils.java ## @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache

[GitHub] [beam] boyuanzz commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
boyuanzz commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434745710 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java ## @@ -906,19 +955,110 @@ public void

[GitHub] [beam] aromanenko-dev commented on pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
aromanenko-dev commented on pull request #11749: URL: https://github.com/apache/beam/pull/11749#issuecomment-638348082 Also, please, take a look on SpotBugs issues. This is an automated message from the Apache Git Service.

[GitHub] [beam] aromanenko-dev commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
aromanenko-dev commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434717019 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java ## @@ -817,6 +847,24 @@ public void

[GitHub] [beam] boyuanzz commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
boyuanzz commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434739659 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaViaSDF.java ## @@ -0,0 +1,697 @@ +/* + * Licensed to the Apache

[GitHub] [beam] robertwb commented on a change in pull request #11901: Prototype schema-inferring Row constructor.

2020-06-03 Thread GitBox
robertwb commented on a change in pull request #11901: URL: https://github.com/apache/beam/pull/11901#discussion_r434734464 ## File path: sdks/python/apache_beam/typehints/opcodes.py ## @@ -120,7 +120,7 @@ def get_iter(state, unused_arg): def symmetric_binary_op(state,

[GitHub] [beam] robertwb commented on a change in pull request #11901: Prototype schema-inferring Row constructor.

2020-06-03 Thread GitBox
robertwb commented on a change in pull request #11901: URL: https://github.com/apache/beam/pull/11901#discussion_r434734784 ## File path: sdks/python/apache_beam/transforms/sql.py ## @@ -74,3 +77,8 @@ def __init__(self, query, dialect=None):

[GitHub] [beam] robertwb commented on a change in pull request #11835: Various fixes to allow Java PAssert to run on Python

2020-06-03 Thread GitBox
robertwb commented on a change in pull request #11835: URL: https://github.com/apache/beam/pull/11835#discussion_r434730606 ## File path: sdks/python/apache_beam/transforms/trigger_test.py ## @@ -518,6 +519,28 @@ def format_result(k_v): 'B-3': {10, 15, 16},

[GitHub] [beam] aaltay commented on a change in pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-03 Thread GitBox
aaltay commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r434724157 ## File path: .github/workflows/build_wheels.yml ## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: +branches: + - master + -

[GitHub] [beam] tysonjh commented on a change in pull request #11856: [BEAM-7505] SideInput Python Load Test job

2020-06-03 Thread GitBox
tysonjh commented on a change in pull request #11856: URL: https://github.com/apache/beam/pull/11856#discussion_r434721022 ## File path: .test-infra/jenkins/job_LoadTests_SideInput_Python.groovy ## @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [beam] ibzib commented on pull request #11902: [BEAM-10168] Add Github 'publish release' to release guide

2020-06-03 Thread GitBox
ibzib commented on pull request #11902: URL: https://github.com/apache/beam/pull/11902#issuecomment-638332379 > Do you think its worth filing a jira to track that? I don't think so. The release _is_ signed, by the release manager's Apache email, which may or may not be the same as

[GitHub] [beam] mik-laj commented on pull request #11331: [BEAM-9646] Add Google Cloud vision integration transform

2020-06-03 Thread GitBox
mik-laj commented on pull request #11331: URL: https://github.com/apache/beam/pull/11331#issuecomment-638324836  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [beam] TheNeuralBit commented on pull request #11902: [BEAM-10168] Add Github 'publish release' to release guide

2020-06-03 Thread GitBox
TheNeuralBit commented on pull request #11902: URL: https://github.com/apache/beam/pull/11902#issuecomment-638319768 Do you think its worth filing a jira to track that? This is an automated message from the Apache Git

[GitHub] [beam] pabloem commented on pull request #11898: [BEAM-10178] Improved error messages when option is not set

2020-06-03 Thread GitBox
pabloem commented on pull request #11898: URL: https://github.com/apache/beam/pull/11898#issuecomment-638315417 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] pabloem commented on pull request #11893: [BEAM-10175] use bearer token header instead of url param for token

2020-06-03 Thread GitBox
pabloem commented on pull request #11893: URL: https://github.com/apache/beam/pull/11893#issuecomment-638315244 A test seems to be flaky? https://builds.apache.org/job/beam_PostCommit_Java_PR/394/ https://builds.apache.org/job/beam_PostCommit_Java_PR/395/

[GitHub] [beam] pabloem commented on pull request #11893: [BEAM-10175] use bearer token header instead of url param for token

2020-06-03 Thread GitBox
pabloem commented on pull request #11893: URL: https://github.com/apache/beam/pull/11893#issuecomment-638314655 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] pabloem commented on pull request #11896: [BEAM-8841] Make bigquery_avro_tools_test py3 compatible

2020-06-03 Thread GitBox
pabloem commented on pull request #11896: URL: https://github.com/apache/beam/pull/11896#issuecomment-638314463 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] ibzib merged pull request #11907: [BEAM-10096] Fix Spark runners numbering

2020-06-03 Thread GitBox
ibzib merged pull request #11907: URL: https://github.com/apache/beam/pull/11907 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] mwalenia merged pull request #11331: [BEAM-9646] Add Google Cloud vision integration transform

2020-06-03 Thread GitBox
mwalenia merged pull request #11331: URL: https://github.com/apache/beam/pull/11331 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] pabloem commented on pull request #11896: [BEAM-8841] Make bigquery_avro_tools_test py3 compatible

2020-06-03 Thread GitBox
pabloem commented on pull request #11896: URL: https://github.com/apache/beam/pull/11896#issuecomment-638310807 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] chunyang commented on pull request #11896: [BEAM-8841] Make bigquery_avro_tools_test py3 compatible

2020-06-03 Thread GitBox
chunyang commented on pull request #11896: URL: https://github.com/apache/beam/pull/11896#issuecomment-638301892 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] nfisher commented on pull request #11732: [BEAM-10017] Expose Cassandra Connect and Read timeouts

2020-06-03 Thread GitBox
nfisher commented on pull request #11732: URL: https://github.com/apache/beam/pull/11732#issuecomment-638299088 @echauchot sorry little crazy with work at the moment. Was hoping to get to the remainder of it this weekend.

[GitHub] [beam] iemejia commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
iemejia commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434666700 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaViaSDF.java ## @@ -0,0 +1,697 @@ +/* + * Licensed to the Apache

[GitHub] [beam] iemejia commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
iemejia commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434666700 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaViaSDF.java ## @@ -0,0 +1,697 @@ +/* + * Licensed to the Apache

[GitHub] [beam] iemejia commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
iemejia commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434666700 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaViaSDF.java ## @@ -0,0 +1,697 @@ +/* + * Licensed to the Apache

[GitHub] [beam] iemejia commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
iemejia commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434666700 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaViaSDF.java ## @@ -0,0 +1,697 @@ +/* + * Licensed to the Apache

[GitHub] [beam] iemejia commented on a change in pull request #11749: [BEAM-9977] Implement ReadFromKafkaViaSDF

2020-06-03 Thread GitBox
iemejia commented on a change in pull request #11749: URL: https://github.com/apache/beam/pull/11749#discussion_r434666700 ## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaViaSDF.java ## @@ -0,0 +1,697 @@ +/* + * Licensed to the Apache

[GitHub] [beam] kamilwu commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests

2020-06-03 Thread GitBox
kamilwu commented on pull request #11909: URL: https://github.com/apache/beam/pull/11909#issuecomment-638261657 R: @iemejia @mxm Let me know what do you think. I put a link to dashboards in the description. Our InfluxDB has yearly retention policy, which means data older than one

[GitHub] [beam] kamilwu opened a new pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests

2020-06-03 Thread GitBox
kamilwu opened a new pull request #11909: URL: https://github.com/apache/beam/pull/11909 Adds Grafana dashboards for Nexmark tests. Dashboards can be previewed on http://metrics.beam.apache.org Thank you for your contribution! Follow this checklist

[GitHub] [beam] jaketf commented on pull request #11893: [BEAM-10175] use bearer token header instead of url param for token

2020-06-03 Thread GitBox
jaketf commented on pull request #11893: URL: https://github.com/apache/beam/pull/11893#issuecomment-638253812 looks like post commit got stuck in the queue This is an automated message from the Apache Git Service. To

[GitHub] [beam] TobKed edited a comment on pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-03 Thread GitBox
TobKed edited a comment on pull request #11877: URL: https://github.com/apache/beam/pull/11877#issuecomment-638238507 cc @kamilwu @damgad @aaltay @tysonjh This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] TobKed commented on pull request #11877: [BEAM-10184] Build python wheels on GitHub Actions for Linux/MacOS

2020-06-03 Thread GitBox
TobKed commented on pull request #11877: URL: https://github.com/apache/beam/pull/11877#issuecomment-638238507 cc @kamilwu @damgad @aaltay This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] echauchot commented on pull request #11732: [BEAM-10017] Expose Cassandra Connect and Read timeouts

2020-06-03 Thread GitBox
echauchot commented on pull request #11732: URL: https://github.com/apache/beam/pull/11732#issuecomment-638214566 @nfisher you seem to lack time, don't worry I'll do the final touches. I will manually merge your commit to master alongside with one of my own and close the PR

[GitHub] [beam] rmannibucau commented on pull request #11886: [BEAM-8647] delete .mailmap

2020-06-03 Thread GitBox
rmannibucau commented on pull request #11886: URL: https://github.com/apache/beam/pull/11886#issuecomment-638200730 @iemejia yes and no, 1. it is quite clear in europe and affects US based entities + 2 ASF can't justify to need this file (side note: it can trivially be replaced by GH API

[GitHub] [beam] mxm commented on a change in pull request #11889: [BEAM-10164] Flink Batch Runner: Memory efficient combine implementat…

2020-06-03 Thread GitBox
mxm commented on a change in pull request #11889: URL: https://github.com/apache/beam/pull/11889#discussion_r434568958 ## File path: runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkBatchTransformTranslators.java ## @@ -476,16 +476,15 @@ public void

[GitHub] [beam] mxm merged pull request #11889: [BEAM-10164] Flink Batch Runner: Memory efficient combine implementat…

2020-06-03 Thread GitBox
mxm merged pull request #11889: URL: https://github.com/apache/beam/pull/11889 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [beam] iemejia commented on pull request #11886: [BEAM-8647] delete .mailmap

2020-06-03 Thread GitBox
iemejia commented on pull request #11886: URL: https://github.com/apache/beam/pull/11886#issuecomment-638194592 For extra info the discussion on `privacy-discuss@` on how to handle these cases (conclusion nothing is clear and we do not have real ASF recommendations yet)

[GitHub] [beam] iemejia merged pull request #11908: [BEAM-9948] Fix mascot invalid directory link

2020-06-03 Thread GitBox
iemejia merged pull request #11908: URL: https://github.com/apache/beam/pull/11908 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] iemejia opened a new pull request #11908: [BEAM-9948] Fix mascot invalid directory link

2020-06-03 Thread GitBox
iemejia opened a new pull request #11908: URL: https://github.com/apache/beam/pull/11908 R: @mwalenia This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [beam] iemejia commented on pull request #11908: [BEAM-9948] Fix mascot invalid directory link

2020-06-03 Thread GitBox
iemejia commented on pull request #11908: URL: https://github.com/apache/beam/pull/11908#issuecomment-638176382 Staged site: http://apache-beam-website-pull-requests.storage.googleapis.com/11908/community/mascot/index.html

  1   2   >