[GitHub] [beam] TobKed commented on pull request #12222: Gate GH actions so it only runs in main/upstream repo

2020-07-13 Thread GitBox
TobKed commented on pull request #1: URL: https://github.com/apache/beam/pull/1#issuecomment-658002752 I checked actions tab in repository which I recently forked and I got information and question about enabling workflows. ![Workflows aren’t being run on this forked repository.

[GitHub] [beam] damondouglas commented on pull request #12206: [Beam-10428] Consolidate Go SDK katas as a single module

2020-07-13 Thread GitBox
damondouglas commented on pull request #12206: URL: https://github.com/apache/beam/pull/12206#issuecomment-657962062 Thank you, Henry. I'll keep going then. I agree with you about the casing preference. @lostluck are you ok with the module name `apache.org/beam/learning/katas`? -

[GitHub] [beam] boyuanzz commented on pull request #12241: [BEAM-10420] Fix minor race condition related to splitting that will cause None has no method 'yyy'

2020-07-13 Thread GitBox
boyuanzz commented on pull request #12241: URL: https://github.com/apache/beam/pull/12241#issuecomment-657960214 Could you please explain more about the race condition here? This is an automated message from the Apache Git Se

[GitHub] [beam] boyuanzz commented on pull request #12016: [BEAM-10341] Support drain in python and java SDF

2020-07-13 Thread GitBox
boyuanzz commented on pull request #12016: URL: https://github.com/apache/beam/pull/12016#issuecomment-657956576 > Note to self, make sure that capabilities are added for SDKs that support drain. Could you please point me to the place of adding capabilities? ---

[GitHub] [beam] boyuanzz commented on a change in pull request #12016: [BEAM-10341] Support drain in python and java SDF

2020-07-13 Thread GitBox
boyuanzz commented on a change in pull request #12016: URL: https://github.com/apache/beam/pull/12016#discussion_r454089322 ## File path: sdks/python/apache_beam/runners/worker/operations.py ## @@ -743,11 +743,36 @@ def pcollection_count_monitoring_infos(self, tag_to_pcollecti

[GitHub] [beam] boyuanzz commented on a change in pull request #12016: [BEAM-10341] Support drain in python and java SDF

2020-07-13 Thread GitBox
boyuanzz commented on a change in pull request #12016: URL: https://github.com/apache/beam/pull/12016#discussion_r454081616 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/reflect/DoFnInvokersTest.java ## @@ -742,10 +785,202 @@ public void outputWithT

[GitHub] [beam] henryken commented on pull request #12206: [Beam-10428] Consolidate Go SDK katas as a single module

2020-07-13 Thread GitBox
henryken commented on pull request #12206: URL: https://github.com/apache/beam/pull/12206#issuecomment-657949086 Thanks for trying this out @damondouglas. It looks good to me and I tested it and it works just fine. It's unfortunate that we have to use lowercase and underscore for t

[GitHub] [beam] boyuanzz commented on a change in pull request #12016: [BEAM-10341] Support drain in python and java SDF

2020-07-13 Thread GitBox
boyuanzz commented on a change in pull request #12016: URL: https://github.com/apache/beam/pull/12016#discussion_r454081206 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/reflect/DoFnInvokersTest.java ## @@ -742,10 +785,202 @@ public void outputWithT

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

2020-07-13 Thread GitBox
ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454079645 ## File path: sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/TestInput.java ## @@ -258,6 +259,14 @@

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

2020-07-13 Thread GitBox
ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454079645 ## File path: sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/TestInput.java ## @@ -258,6 +259,14 @@

[GitHub] [beam] boyuanzz commented on a change in pull request #12016: [BEAM-10341] Support drain in python and java SDF

2020-07-13 Thread GitBox
boyuanzz commented on a change in pull request #12016: URL: https://github.com/apache/beam/pull/12016#discussion_r454077833 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/RestrictionTracker.java ## @@ -99,6 +99,24 @@ */ public a

[GitHub] [beam] boyuanzz commented on a change in pull request #12016: [BEAM-10341] Support drain in python and java SDF

2020-07-13 Thread GitBox
boyuanzz commented on a change in pull request #12016: URL: https://github.com/apache/beam/pull/12016#discussion_r454073301 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/RestrictionTracker.java ## @@ -99,6 +99,24 @@ */ public a

[GitHub] [beam] boyuanzz commented on a change in pull request #12016: [BEAM-10341] Support drain in python and java SDF

2020-07-13 Thread GitBox
boyuanzz commented on a change in pull request #12016: URL: https://github.com/apache/beam/pull/12016#discussion_r454070888 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/GrowableOffsetRangeTracker.java ## @@ -140,4 +140,15 @@ public P

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

2020-07-13 Thread GitBox
ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454053171 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BeamBigQuerySqlDialect.java ## @@ -156

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

2020-07-13 Thread GitBox
ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454045168 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ExpressionConverter.java ## @@ -8

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

2020-07-13 Thread GitBox
ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454044979 ## File path: sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlDialectSpecTest.java ## @@ -2174,6 +21

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

2020-07-13 Thread GitBox
ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454044618 ## File path: sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlDialectSpecTest.java ## @@ -2174,6 +21

[GitHub] [beam] ibzib opened a new pull request #12250: [BEAM-10460] Limit SparkPortableExecutionTest to a single thread.

2020-07-13 Thread GitBox
ibzib opened a new pull request #12250: URL: https://github.com/apache/beam/pull/12250 Starting up multiple Spark clusters at a time uses a lot of memory. Also, using a thread pool (even an artificially limited thread pool) is closer to the actual usage of JobInvocation than direct executo

[GitHub] [beam] emilymye commented on pull request #12222: Gate GH actions so it only runs in main/upstream repo

2020-07-13 Thread GitBox
emilymye commented on pull request #1: URL: https://github.com/apache/beam/pull/1#issuecomment-657909594 I'm sure @TobKed knows more about Github actions than I do, but it seems there isn't a common convention. Based on the docs [docs](https://docs.github.com/en/actions/getting-sta

[GitHub] [beam] rosetn commented on pull request #12248: Cleanup some old pipeline examples.

2020-07-13 Thread GitBox
rosetn commented on pull request #12248: URL: https://github.com/apache/beam/pull/12248#issuecomment-657909633 R: @davidcavazos This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] apilloud merged pull request #12238: [BEAM-4709] Adding PostCommit jenkins job that builds Javadoc

2020-07-13 Thread GitBox
apilloud merged pull request #12238: URL: https://github.com/apache/beam/pull/12238 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] apilloud commented on pull request #12247: [BEAM-9179] Remove dead code: Java implementation of ZetaSQL functions

2020-07-13 Thread GitBox
apilloud commented on pull request #12247: URL: https://github.com/apache/beam/pull/12247#issuecomment-657905365 I would be fine with deleting anything that is obviously broken and cleanup, but this isn't dead code. I know of at least one customer use case where they are disabling `BeamZet

[GitHub] [beam] rosetn commented on a change in pull request #12246: Add Python snippets in testing guide.

2020-07-13 Thread GitBox
rosetn commented on a change in pull request #12246: URL: https://github.com/apache/beam/pull/12246#discussion_r454032716 ## File path: website/www/site/content/en/documentation/pipelines/test-your-pipeline.md ## @@ -180,8 +191,21 @@ PAssert.that(output) "elem3", "elem2"

[GitHub] [beam] KevinGG opened a new pull request #12249: Make tracking/cleanup of cache and in-environment states per pipeline

2020-07-13 Thread GitBox
KevinGG opened a new pull request #12249: URL: https://github.com/apache/beam/pull/12249 1. Interactive Beam now creates 1 cache manager for each user defined pipeline. 2. Cache managers are created lazily when they are used. 3. Cleanup of cache and states are carried out independentl

[GitHub] [beam] apilloud commented on pull request #12238: [BEAM-4709] Adding PostCommit jenkins job that builds Javadoc

2020-07-13 Thread GitBox
apilloud commented on pull request #12238: URL: https://github.com/apache/beam/pull/12238#issuecomment-657896739 Run Javadoc PostCommit This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [beam] davidcavazos commented on pull request #12217: [BEAM-7390] Add cogroupbykey code snippets

2020-07-13 Thread GitBox
davidcavazos commented on pull request #12217: URL: https://github.com/apache/beam/pull/12217#issuecomment-657896586 R: @aaltay This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] rosetn commented on pull request #12217: [BEAM-7390] Add cogroupbykey code snippets

2020-07-13 Thread GitBox
rosetn commented on pull request #12217: URL: https://github.com/apache/beam/pull/12217#issuecomment-657894258 LGTM. Thank you, David! This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [beam] pabloem commented on a change in pull request #12074: [BEAM-10313] Add without_defaults to built-in Combiners in Python

2020-07-13 Thread GitBox
pabloem commented on a change in pull request #12074: URL: https://github.com/apache/beam/pull/12074#discussion_r447317739 ## File path: sdks/python/apache_beam/transforms/combiners.py ## @@ -62,26 +62,28 @@ TimestampType = Union[int, float, Timestamp, Duration] +class Com

[GitHub] [beam] robertwb opened a new pull request #12248: Cleanup some old pipeline examples.

2020-07-13 Thread GitBox
robertwb opened a new pull request #12248: URL: https://github.com/apache/beam/pull/12248 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://b

[GitHub] [beam] robertwb commented on pull request #12248: Cleanup some old pipeline examples.

2020-07-13 Thread GitBox
robertwb commented on pull request #12248: URL: https://github.com/apache/beam/pull/12248#issuecomment-657884514 R: @rosetn This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [beam] robinyqiu commented on pull request #12247: [BEAM-9179] Remove dead code: Java implementation of ZetaSQL functions

2020-07-13 Thread GitBox
robinyqiu commented on pull request #12247: URL: https://github.com/apache/beam/pull/12247#issuecomment-657880976 I moved `SqlOperators` to the `translation` sub-directory and changed its content. For some reason GitHub does not recognize that. Instead it treats the change as a file deleti

[GitHub] [beam] robinyqiu opened a new pull request #12247: [BEAM-9179] Remove dead code: Java implementation of ZetaSQL functions

2020-07-13 Thread GitBox
robinyqiu opened a new pull request #12247: URL: https://github.com/apache/beam/pull/12247 After we switched to `BeamZetaSqlCalcRel`, the Java function implementation is no longer needed. But the code stayed there without being used, so I am removing them now, along with some other unused

[GitHub] [beam] apilloud commented on pull request #12238: [BEAM-4709] Adding PostCommit jenkins job that builds Javadoc

2020-07-13 Thread GitBox
apilloud commented on pull request #12238: URL: https://github.com/apache/beam/pull/12238#issuecomment-657875503 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] robertwb opened a new pull request #12246: Add Python snippets in testing guide.

2020-07-13 Thread GitBox
robertwb opened a new pull request #12246: URL: https://github.com/apache/beam/pull/12246 This recently came up on the users list, and there was no good documentation to be found. Thank you for your contribution! Follow this checklist to help us incor

[GitHub] [beam] robertwb commented on pull request #12246: Add Python snippets in testing guide.

2020-07-13 Thread GitBox
robertwb commented on pull request #12246: URL: https://github.com/apache/beam/pull/12246#issuecomment-657873436 R: @rosetn This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [beam] amaliujia commented on a change in pull request #12232: [Beam-9543] Support Match Recognition in Beam SQL

2020-07-13 Thread GitBox
amaliujia commented on a change in pull request #12232: URL: https://github.com/apache/beam/pull/12232#discussion_r453996803 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamMatchRel.java ## @@ -0,0 +1,380 @@ +/* + * Licensed

[GitHub] [beam] amaliujia commented on a change in pull request #12232: [Beam-9543] Support Match Recognition in Beam SQL

2020-07-13 Thread GitBox
amaliujia commented on a change in pull request #12232: URL: https://github.com/apache/beam/pull/12232#discussion_r453995446 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/cep/CEPTypeName.java ## @@ -0,0 +1,33 @@ +/* + * Licensed t

[GitHub] [beam] robinyqiu commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

2020-07-13 Thread GitBox
robinyqiu commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r453995996 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BeamBigQuerySqlDialect.java ## @@ -156,7

[GitHub] [beam] robertwb commented on pull request #12245: DoFnTester is deprecated and should not be recommended.

2020-07-13 Thread GitBox
robertwb commented on pull request #12245: URL: https://github.com/apache/beam/pull/12245#issuecomment-657851028 R: @rosetn This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [beam] robertwb opened a new pull request #12245: DoFnTester is deprecated and should not be recommended.

2020-07-13 Thread GitBox
robertwb opened a new pull request #12245: URL: https://github.com/apache/beam/pull/12245 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://b

[GitHub] [beam] aaltay merged pull request #12244: add direct runner xlang test badge

2020-07-13 Thread GitBox
aaltay merged pull request #12244: URL: https://github.com/apache/beam/pull/12244 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [beam] aaltay commented on pull request #12222: Gate GH actions so it only runs in main/upstream repo

2020-07-13 Thread GitBox
aaltay commented on pull request #1: URL: https://github.com/apache/beam/pull/1#issuecomment-657842821 I agree with both of you. Keeping tests by default would allow easy testing. (But is it free for all even if theirs forks are private?) And making it opt-in sounds like least disr

[GitHub] [beam] aaltay commented on a change in pull request #12172: [BEAM-10185] Building python wheels for Windows (on Github Actions)

2020-07-13 Thread GitBox
aaltay commented on a change in pull request #12172: URL: https://github.com/apache/beam/pull/12172#discussion_r453991538 ## File path: sdks/python/apache_beam/runners/worker/crossplatform_time.h ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [beam] kennknowles merged pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
kennknowles merged pull request #12216: URL: https://github.com/apache/beam/pull/12216 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] kennknowles commented on pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
kennknowles commented on pull request #12216: URL: https://github.com/apache/beam/pull/12216#issuecomment-657837672 Ah, green. This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] kennknowles commented on pull request #12224: [BEAM-10080] Fixes PipelineOptionsFactoryTest and ReflectHelpersTest for Java 11

2020-07-13 Thread GitBox
kennknowles commented on pull request #12224: URL: https://github.com/apache/beam/pull/12224#issuecomment-657837441 Finally green! This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [beam] kennknowles merged pull request #12225: [BEAM-10080] Fix LocalFileSystemsTest to work with Java 11

2020-07-13 Thread GitBox
kennknowles merged pull request #12225: URL: https://github.com/apache/beam/pull/12225 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] kennknowles commented on pull request #12225: [BEAM-10080] Fix LocalFileSystemsTest to work with Java 11

2020-07-13 Thread GitBox
kennknowles commented on pull request #12225: URL: https://github.com/apache/beam/pull/12225#issuecomment-657835783 Finally green! This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [beam] ihji commented on pull request #12244: add direct runner xlang test badge

2020-07-13 Thread GitBox
ihji commented on pull request #12244: URL: https://github.com/apache/beam/pull/12244#issuecomment-657826568 R: @aaltay This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [beam] ihji opened a new pull request #12244: add direct runner xlang test badge

2020-07-13 Thread GitBox
ihji opened a new pull request #12244: URL: https://github.com/apache/beam/pull/12244 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly a

[GitHub] [beam] robinyqiu commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

2020-07-13 Thread GitBox
robinyqiu commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r453958885 ## File path: sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/TestInput.java ## @@ -258,6 +259,14 @@

[GitHub] [beam] amaliujia commented on a change in pull request #12232: [Beam-9543] Support Match Recognition in Beam SQL

2020-07-13 Thread GitBox
amaliujia commented on a change in pull request #12232: URL: https://github.com/apache/beam/pull/12232#discussion_r453979800 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamMatchRel.java ## @@ -0,0 +1,380 @@ +/* + * Licensed

[GitHub] [beam] kennknowles commented on pull request #12224: [BEAM-10080] Fixes PipelineOptionsFactoryTest and ReflectHelpersTest for Java 11

2020-07-13 Thread GitBox
kennknowles commented on pull request #12224: URL: https://github.com/apache/beam/pull/12224#issuecomment-657812019 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [beam] kennknowles commented on pull request #12225: [BEAM-10080] Fix LocalFileSystemsTest to work with Java 11

2020-07-13 Thread GitBox
kennknowles commented on pull request #12225: URL: https://github.com/apache/beam/pull/12225#issuecomment-657808372 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [beam] kennknowles opened a new pull request #12243: [BEAM-10460] Increase SparkPortableExecutionTest.testExecution timeout to 10 minutes

2020-07-13 Thread GitBox
kennknowles opened a new pull request #12243: URL: https://github.com/apache/beam/pull/12243 Successful runs have been seen at 90 seconds. The timeout only needs to catch deadlocks and crashloops and should not be sensitive to performance in either the underlying code or our test infra

[GitHub] [beam] ZijieSong946 commented on pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

2020-07-13 Thread GitBox
ZijieSong946 commented on pull request #12174: URL: https://github.com/apache/beam/pull/12174#issuecomment-657806602 Merge conflict solved. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] kennknowles commented on pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
kennknowles commented on pull request #12216: URL: https://github.com/apache/beam/pull/12216#issuecomment-657799504 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [beam] kennknowles commented on pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
kennknowles commented on pull request #12216: URL: https://github.com/apache/beam/pull/12216#issuecomment-657799415 Generating license report succeeded. The gradle scan shows two flakes: - https://issues.apache.org/jira/browse/BEAM-9187 - https://issues.apache.org/jira/browse/BEA

[GitHub] [beam] saavannanavati commented on a change in pull request #12009: [BEAM-10258] Support type hint annotations on PTransform's expand()

2020-07-13 Thread GitBox
saavannanavati commented on a change in pull request #12009: URL: https://github.com/apache/beam/pull/12009#discussion_r453938415 ## File path: sdks/python/apache_beam/typehints/decorators.py ## @@ -378,6 +378,56 @@ def has_simple_output_type(self): self.output_types a

[GitHub] [beam] saavannanavati commented on a change in pull request #12009: [BEAM-10258] Support type hint annotations on PTransform's expand()

2020-07-13 Thread GitBox
saavannanavati commented on a change in pull request #12009: URL: https://github.com/apache/beam/pull/12009#discussion_r453938415 ## File path: sdks/python/apache_beam/typehints/decorators.py ## @@ -378,6 +378,56 @@ def has_simple_output_type(self): self.output_types a

[GitHub] [beam] kennknowles merged pull request #12227: [BEAM-10081] Fix ApiSurface to work with Java 11

2020-07-13 Thread GitBox
kennknowles merged pull request #12227: URL: https://github.com/apache/beam/pull/12227 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] robinyqiu merged pull request #12231: [BEAM-9178] Support more EXTRACT functions

2020-07-13 Thread GitBox
robinyqiu merged pull request #12231: URL: https://github.com/apache/beam/pull/12231 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] robinyqiu commented on pull request #12231: [BEAM-9178] Support more EXTRACT functions

2020-07-13 Thread GitBox
robinyqiu commented on pull request #12231: URL: https://github.com/apache/beam/pull/12231#issuecomment-657787552 Merging now (tested internally) This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [beam] robinyqiu commented on a change in pull request #12231: [BEAM-9178] Support more EXTRACT functions

2020-07-13 Thread GitBox
robinyqiu commented on a change in pull request #12231: URL: https://github.com/apache/beam/pull/12231#discussion_r453924913 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BeamBigQuerySqlDialect.java ## @@ -359,6

[GitHub] [beam] Hannah-Jiang commented on pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
Hannah-Jiang commented on pull request #12216: URL: https://github.com/apache/beam/pull/12216#issuecomment-657785349 > There was a failure to pull a license from another of the Beam modules because the name was wrong. It was called `fn-execution:2.24.0-SNAPSHOT`. This was already the case

[GitHub] [beam] kennknowles commented on pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
kennknowles commented on pull request #12216: URL: https://github.com/apache/beam/pull/12216#issuecomment-657780781 https://issues.apache.org/jira/browse/BEAM-9187 which said "sickbay" but is clearly running This is an autom

[GitHub] [beam] aaltay commented on pull request #11639: [BEAM-4440] Throw exception when file to stage is not found, instead of logging a warning

2020-07-13 Thread GitBox
aaltay commented on pull request #11639: URL: https://github.com/apache/beam/pull/11639#issuecomment-657779047 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] kennknowles commented on a change in pull request #11792: [BEAM-10435] Add ValidatesRunner task for local_job_service and Java SDK harness

2020-07-13 Thread GitBox
kennknowles commented on a change in pull request #11792: URL: https://github.com/apache/beam/pull/11792#discussion_r453911740 ## File path: runners/portability/java/src/main/java/org/apache/beam/runners/portability/testing/TestUniversalRunner.java ## @@ -0,0 +1,119 @@ +/* + *

[GitHub] [beam] saavannanavati opened a new pull request #12242: [BEAM-10427] Benchmark runtime typechecking for the Python SDK

2020-07-13 Thread GitBox
saavannanavati opened a new pull request #12242: URL: https://github.com/apache/beam/pull/12242 This PR adds a load test for evaluating the performance of the runtime typechecking system for the Python SDK. It works by comparing the performance of a pipeline with the runtime typechecking f

[GitHub] [beam] santhh commented on a change in pull request #11566: [BEAM-9723] Add DLP integration transforms

2020-07-13 Thread GitBox
santhh commented on a change in pull request #11566: URL: https://github.com/apache/beam/pull/11566#discussion_r453907291 ## File path: sdks/java/extensions/ml/src/main/java/org/apache/beam/sdk/extensions/ml/DLPDeidentifyText.java ## @@ -0,0 +1,282 @@ +/* + * Licensed to the A

[GitHub] [beam] youngoli commented on pull request #12046: [BEAM-10290] Add a unit(ish) test for dynamic splitting correctness.

2020-07-13 Thread GitBox
youngoli commented on pull request #12046: URL: https://github.com/apache/beam/pull/12046#issuecomment-657760963 This PR is waiting for the dynamic splitting implementation to go in first, since it's being used to test that. I'm working on that this week so it shouldn't be much longer. -

[GitHub] [beam] kennknowles commented on pull request #11792: [BEAM-10435] Add ValidatesRunner task for local_job_service and Java SDK harness

2020-07-13 Thread GitBox
kennknowles commented on pull request #11792: URL: https://github.com/apache/beam/pull/11792#issuecomment-657760388 Flakes in this run: - https://issues.apache.org/jira/browse/BEAM-10474 failure in py DisplayDataTest And then I just introduced an actual compile error in my fi

[GitHub] [beam] kennknowles commented on pull request #12224: [BEAM-10080] Fixes PipelineOptionsFactoryTest and ReflectHelpersTest for Java 11

2020-07-13 Thread GitBox
kennknowles commented on pull request #12224: URL: https://github.com/apache/beam/pull/12224#issuecomment-657744481 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [beam] kennknowles commented on pull request #12225: [BEAM-10080] Fix LocalFileSystemsTest to work with Java 11

2020-07-13 Thread GitBox
kennknowles commented on pull request #12225: URL: https://github.com/apache/beam/pull/12225#issuecomment-657743373 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [beam] robinyqiu merged pull request #12237: [BEAM-10407] Revamped SchemaCapableIOProvider interface to SchemaIOProvider

2020-07-13 Thread GitBox
robinyqiu merged pull request #12237: URL: https://github.com/apache/beam/pull/12237 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] ffDante commented on pull request #4341: [BEAM-3354] Fixed a bug that prevented processing time timers to be reset

2020-07-13 Thread GitBox
ffDante commented on pull request #4341: URL: https://github.com/apache/beam/pull/4341#issuecomment-657742217 It seems like this is still not working with Dataflow Runner This is an automated message from the Apache Git Servi

[GitHub] [beam] chamikaramj merged pull request #11919: [BEAM-10114] Copy Pub/Sub Lite IO from Pub/Sub Lite github to beam.

2020-07-13 Thread GitBox
chamikaramj merged pull request #11919: URL: https://github.com/apache/beam/pull/11919 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] kennknowles commented on pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
kennknowles commented on pull request #12216: URL: https://github.com/apache/beam/pull/12216#issuecomment-657737628 There was a failure to pull a license from another of the Beam modules because the name was wrong. It was called `fn-execution:2.24.0-SNAPSHOT`. This was already the case and

[GitHub] [beam] kennknowles commented on pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
kennknowles commented on pull request #12216: URL: https://github.com/apache/beam/pull/12216#issuecomment-657736077 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [beam] kennknowles commented on pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
kennknowles commented on pull request #12216: URL: https://github.com/apache/beam/pull/12216#issuecomment-657735289 Filed https://issues.apache.org/jira/browse/BEAM-10470 to track the flake in https://ci-beam.apache.org/job/beam_PreCommit_Java_Commit/12326/testReport/junit/org.apache.beam.s

[GitHub] [beam] aaltay commented on pull request #12233: [BEAM-10465] Add checksum for Python source distribution and wheels

2020-07-13 Thread GitBox
aaltay commented on pull request #12233: URL: https://github.com/apache/beam/pull/12233#issuecomment-657728399 Do we expect workflows to run when there is a change to workflows? This is an automated message from the Apache Gi

[GitHub] [beam] pabloem commented on a change in pull request #12151: [BEAM-9896] Add streaming for SnowflakeIO.Write to Java SDK

2020-07-13 Thread GitBox
pabloem commented on a change in pull request #12151: URL: https://github.com/apache/beam/pull/12151#discussion_r453845146 ## File path: sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/services/SnowflakeStreamingServiceImpl.java ## @@ -0,0 +1,68 @@ +/* +

[GitHub] [beam] lukecwik commented on pull request #12241: [BEAM-10420] Fix minor race condition related to splitting that will cause None has no method 'yyy'

2020-07-13 Thread GitBox
lukecwik commented on pull request #12241: URL: https://github.com/apache/beam/pull/12241#issuecomment-657716085 R: @boyuanzz This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] lukecwik opened a new pull request #12241: [BEAM-10420] Fix minor race condition related to splitting that will cause None has no method 'yyy'

2020-07-13 Thread GitBox
lukecwik opened a new pull request #12241: URL: https://github.com/apache/beam/pull/12241 This will also be important to have this lock when splitting/getting progress for window observing splittable DoFns. Thank you for your contribution! Follow this

[GitHub] [beam] kennknowles commented on pull request #12224: [BEAM-10080] Fixes PipelineOptionsFactoryTest and ReflectHelpersTest for Java 11

2020-07-13 Thread GitBox
kennknowles commented on pull request #12224: URL: https://github.com/apache/beam/pull/12224#issuecomment-657702113 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [beam] kennknowles commented on a change in pull request #12225: [BEAM-10080] Fix LocalFileSystemsTest to work with Java 11

2020-07-13 Thread GitBox
kennknowles commented on a change in pull request #12225: URL: https://github.com/apache/beam/pull/12225#discussion_r453818020 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/LocalFileSystem.java ## @@ -90,9 +91,14 @@ @Override protected List match(Li

[GitHub] [beam] robinyqiu commented on a change in pull request #12237: [BEAM-10407] Revamped SchemaCapableIOProvider interface to SchemaIOProvider

2020-07-13 Thread GitBox
robinyqiu commented on a change in pull request #12237: URL: https://github.com/apache/beam/pull/12237#discussion_r453821984 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubSchemaCapableIOProvider.java ## @@ -116,11 +116,2

[GitHub] [beam] apilloud commented on pull request #12240: [BEAM-10462] transforms handle Double.NaN

2020-07-13 Thread GitBox
apilloud commented on pull request #12240: URL: https://github.com/apache/beam/pull/12240#issuecomment-657697498 R: @lukecwik This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] apilloud opened a new pull request #12240: [BEAM-10462] transforms handle Double.NaN

2020-07-13 Thread GitBox
apilloud opened a new pull request #12240: URL: https://github.com/apache/beam/pull/12240 NaN values currently cause data corruption for Min() and Max(). Fix and add tests. Thank you for your contribution! Follow this checklist to help us incorporate

[GitHub] [beam] kennknowles commented on pull request #12216: For Java container, generate license report only for container modules

2020-07-13 Thread GitBox
kennknowles commented on pull request #12216: URL: https://github.com/apache/beam/pull/12216#issuecomment-657694727 Rebasing to squash in fixup commit, then will merge. This is an automated message from the Apache Git Service

[GitHub] [beam] emilymye commented on pull request #12222: Gate GH actions so it only runs in main/upstream repo

2020-07-13 Thread GitBox
emilymye commented on pull request #1: URL: https://github.com/apache/beam/pull/1#issuecomment-657689639 I think my reasoning was that actions should be specifically opt-in, since disabling them per-fork still requires a manual opt-out step. I guess I'm not entirely sure why these

[GitHub] [beam] lazylynx opened a new pull request #12239: [BEAM-9980] tests tied with Python versions configurable

2020-07-13 Thread GitBox
lazylynx opened a new pull request #12239: URL: https://github.com/apache/beam/pull/12239 Made it possible for some tests to configure which python version to use with gradle properties Thank you for your contribution! Follow this checklist to help us

[GitHub] [beam] robertwb commented on pull request #11963: Add relational GroupBy transform to Python.

2020-07-13 Thread GitBox
robertwb commented on pull request #11963: URL: https://github.com/apache/beam/pull/11963#issuecomment-657688336 I've refactored this to use named tuples rather than rows. PTAL. This is an automated message from the Apache Gi

[GitHub] [beam] damgad commented on pull request #12238: [BEAM-4709] Adding PostCommit jenkins job that builds Javadoc

2020-07-13 Thread GitBox
damgad commented on pull request #12238: URL: https://github.com/apache/beam/pull/12238#issuecomment-657685437 R: @aaltay This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [beam] damgad commented on a change in pull request #12172: [BEAM-10185] Building python wheels for Windows (on Github Actions)

2020-07-13 Thread GitBox
damgad commented on a change in pull request #12172: URL: https://github.com/apache/beam/pull/12172#discussion_r453805179 ## File path: sdks/python/apache_beam/runners/worker/crossplatform_time.h ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [beam] damgad opened a new pull request #12238: [BEAM-4709] Adding PostCommit jenkins job that builds Javadoc

2020-07-13 Thread GitBox
damgad opened a new pull request #12238: URL: https://github.com/apache/beam/pull/12238 Adding PostCommit jenkins job that builds Javadoc Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily

[GitHub] [beam] TobKed commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-13 Thread GitBox
TobKed commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r453790997 ## File path: release/src/main/scripts/download_github_actions_artifacts.py ## @@ -0,0 +1,234 @@ +# +# Licensed to the Apache Software Foundation (ASF) unde

[GitHub] [beam] pabloem commented on pull request #12201: [BEAM-10291] log full thread dump only when lull duration is more tha…

2020-07-13 Thread GitBox
pabloem commented on pull request #12201: URL: https://github.com/apache/beam/pull/12201#issuecomment-657662013 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [beam] sclukas77 opened a new pull request #12237: [BEAM-10407] Revamped SchemaCapableIOProvider interface to SchemaIOProvider

2020-07-13 Thread GitBox
sclukas77 opened a new pull request #12237: URL: https://github.com/apache/beam/pull/12237 Revamped SchemaCapableIOProvider to SchemaIOProvider in anticipation of schema aware IOs, which, unlike SchemaCapableIOProviders, do not require a data schema passed from the user. A requiresDataSche

[GitHub] [beam] lukecwik merged pull request #12226: [BEAM-10450] [BEAM-10449] Fix empty flatten bundle processing.

2020-07-13 Thread GitBox
lukecwik merged pull request #12226: URL: https://github.com/apache/beam/pull/12226 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

  1   2   >