[GitHub] [beam] lukecwik commented on a change in pull request #12260: [BEAM-10486] Update OffsetRestrictionTracker to honor failed claim attempt marking the claim as done when checkpointing.

lukecwik commented on a change in pull request #12260: URL: https://github.com/apache/beam/pull/12260#discussion_r454803546 ## File path: sdks/python/apache_beam/io/restriction_trackers.py ## @@ -92,7 +92,8 @@ def __init__(self, offset_range): self._checkpointed = False

[GitHub] [beam] lukecwik commented on a change in pull request #12260: [BEAM-10486] Update OffsetRestrictionTracker to honor failed claim attempt marking the claim as done when checkpointing.

lukecwik commented on a change in pull request #12260: URL: https://github.com/apache/beam/pull/12260#discussion_r454798710 ## File path: sdks/python/apache_beam/io/restriction_trackers.py ## @@ -92,7 +92,8 @@ def __init__(self, offset_range): self._checkpointed = False

[GitHub] [beam] lukecwik commented on pull request #12260: [BEAM-10486] Update OffsetRestrictionTracker to honor failed claim attempt marking the claim as done when checkpointing.

lukecwik commented on pull request #12260: URL: https://github.com/apache/beam/pull/12260#issuecomment-658551949 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] lukecwik commented on pull request #12260: [BEAM-10486] Update OffsetRestrictionTracker to honor failed claim attempt marking the claim as done when checkpointing.

lukecwik commented on pull request #12260: URL: https://github.com/apache/beam/pull/12260#issuecomment-658551676 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] boyuanzz commented on a change in pull request #12260: [BEAM-10486] Update OffsetRestrictionTracker to honor failed claim attempt marking the claim as done when checkpointing.

boyuanzz commented on a change in pull request #12260: URL: https://github.com/apache/beam/pull/12260#discussion_r454785667 ## File path: sdks/python/apache_beam/io/restriction_trackers.py ## @@ -92,7 +92,8 @@ def __init__(self, offset_range): self._checkpointed = False

[GitHub] [beam] kennknowles commented on pull request #11639: [BEAM-4440] Throw exception when file to stage is not found, instead of logging a warning

kennknowles commented on pull request #11639: URL: https://github.com/apache/beam/pull/11639#issuecomment-658521333 Finally green! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] kennknowles merged pull request #11639: [BEAM-4440] Throw exception when file to stage is not found, instead of logging a warning

kennknowles merged pull request #11639: URL: https://github.com/apache/beam/pull/11639 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] lukecwik commented on pull request #12262: [BEAM-10491] Simplify PeriodicSequence generator to use OffsetRanges with whole whole numbers

lukecwik commented on pull request #12262: URL: https://github.com/apache/beam/pull/12262#issuecomment-658503845 R: @Ardagan @pabloem I wanted to simplify the existing implementation because users will look at existing code to see how to use it. Note that the tests won't pass with

[GitHub] [beam] lukecwik opened a new pull request #12262: [BEAM-10491] Simplify PeriodicSequence generator to use OffsetRanges with whole whole numbers

lukecwik opened a new pull request #12262: URL: https://github.com/apache/beam/pull/12262 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose

[GitHub] [beam] robinyqiu opened a new pull request #12261: [BEAM-10490] Support read/write ZetaSQL DATE/TIME types from/to BigQuery

robinyqiu opened a new pull request #12261: URL: https://github.com/apache/beam/pull/12261 Also fix reading BYTES from BigQuery which is also failing currently. r: @kennknowles @chamikaramj Post-Commit Tests Status (on master branch)

[GitHub] [beam] lukecwik commented on pull request #12260: [BEAM-10486] Update OffsetRestrictionTracker to honor failed claim attempt marking the claim as done when checkpointing.

lukecwik commented on pull request #12260: URL: https://github.com/apache/beam/pull/12260#issuecomment-658501584 R: @boyuanzz This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] lukecwik opened a new pull request #12260: [BEAM-10486] Update OffsetRestrictionTracker to honor failed claim attempt marking the claim as done when checkpointing.

lukecwik opened a new pull request #12260: URL: https://github.com/apache/beam/pull/12260 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose

[GitHub] [beam] saavannanavati commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit

saavannanavati commented on a change in pull request #12257: URL: https://github.com/apache/beam/pull/12257#discussion_r454738417 ## File path: sdks/python/setup.py ## @@ -193,6 +193,8 @@ def get_version(): 'requests_mock>=1.7,<2.0', 'tenacity>=5.0.2,<6.0',

[GitHub] [beam] saavannanavati commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit

saavannanavati commented on a change in pull request #12257: URL: https://github.com/apache/beam/pull/12257#discussion_r454738417 ## File path: sdks/python/setup.py ## @@ -193,6 +193,8 @@ def get_version(): 'requests_mock>=1.7,<2.0', 'tenacity>=5.0.2,<6.0',

[GitHub] [beam] saavannanavati commented on a change in pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit

saavannanavati commented on a change in pull request #12257: URL: https://github.com/apache/beam/pull/12257#discussion_r454738417 ## File path: sdks/python/setup.py ## @@ -193,6 +193,8 @@ def get_version(): 'requests_mock>=1.7,<2.0', 'tenacity>=5.0.2,<6.0',

[GitHub] [beam] amaliujia commented on pull request #11975: [BEAM-9198] BeamSQL aggregation analytics functionality

amaliujia commented on pull request #11975: URL: https://github.com/apache/beam/pull/11975#issuecomment-658485305 @jhnmora000 thanks for you contribution! This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] amaliujia merged pull request #11975: [BEAM-9198] BeamSQL aggregation analytics functionality

amaliujia merged pull request #11975: URL: https://github.com/apache/beam/pull/11975 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] robertwb opened a new pull request #12259: [BEAM-9561] Run Pandas tests on Beam.

robertwb opened a new pull request #12259: URL: https://github.com/apache/beam/pull/12259 Few tests are actually run at this point, but we can enable them as implementations are added and/or validated. Thank you for your contribution! Follow this

[GitHub] [beam] aaltay merged pull request #12249: Make tracking/cleanup of cache and in-environment states per pipeline

aaltay merged pull request #12249: URL: https://github.com/apache/beam/pull/12249 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] amaliujia commented on pull request #11975: [BEAM-9198] BeamSQL aggregation analytics functionality

amaliujia commented on pull request #11975: URL: https://github.com/apache/beam/pull/11975#issuecomment-658473749 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] robertwb commented on a change in pull request #12009: [BEAM-10258] Support type hint annotations on PTransform's expand()

robertwb commented on a change in pull request #12009: URL: https://github.com/apache/beam/pull/12009#discussion_r454713524 ## File path: website/www/site/content/en/documentation/sdks/python-type-safety.md ## @@ -90,6 +91,15 @@ The following code declares an `int` input and a

[GitHub] [beam] apilloud commented on pull request #12240: [BEAM-10462] transforms handle Double.NaN

apilloud commented on pull request #12240: URL: https://github.com/apache/beam/pull/12240#issuecomment-658468710 I went to find you some documentation and found the `java.lang.Math` library, which I updated it to use:

[GitHub] [beam] KevinGG commented on a change in pull request #12249: Make tracking/cleanup of cache and in-environment states per pipeline

KevinGG commented on a change in pull request #12249: URL: https://github.com/apache/beam/pull/12249#discussion_r454707919 ## File path: sdks/python/apache_beam/runners/interactive/pipeline_instrument.py ## @@ -556,19 +556,26 @@ def _process(self, pcoll): if not

[GitHub] [beam] amaliujia commented on pull request #11975: [BEAM-9198] BeamSQL aggregation analytics functionality

amaliujia commented on pull request #11975: URL: https://github.com/apache/beam/pull/11975#issuecomment-658463117 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] amaliujia commented on pull request #11975: [BEAM-9198] BeamSQL aggregation analytics functionality

amaliujia commented on pull request #11975: URL: https://github.com/apache/beam/pull/11975#issuecomment-658460751 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] amaliujia commented on pull request #11967: [BEAM-9992] | use Sets transform in BeamSQL

amaliujia commented on pull request #11967: URL: https://github.com/apache/beam/pull/11967#issuecomment-658460380 Ok it seems a `NULL` value handling problem (causing a crashing on program execution). Can you log the java execution stack in BEAM-9992 and we can continue discussion

[GitHub] [beam] iemejia commented on pull request #12075: Remove usage of jdk-internal NotNull annotation

iemejia commented on pull request #12075: URL: https://github.com/apache/beam/pull/12075#issuecomment-658460104 Maybe worth to create a rule that restrict both imports similar to

[GitHub] [beam] iemejia edited a comment on pull request #12075: Remove usage of jdk-internal NotNull annotation

iemejia edited a comment on pull request #12075: URL: https://github.com/apache/beam/pull/12075#issuecomment-658458939 @nielm any chance you can open a new PR and grep the rest of the codebase but for `com.sun.istack.Nullable`, it seems there are occurrences of a similar issue in other

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454663699 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ExpressionConverter.java ## @@

[GitHub] [beam] iemejia commented on pull request #12075: Remove usage of jdk-internal NotNull annotation

iemejia commented on pull request #12075: URL: https://github.com/apache/beam/pull/12075#issuecomment-658458939 @nielm any chance you can open a new PR and fully grep the rest of the codebase, it seems there are occurrences of the same issue in other modules (spark and hcataog).

[GitHub] [beam] lukecwik opened a new pull request #12258: [WIP][BEAM-10486] Fix splitting logic to not return the wrong range after a failed try claim attempt.

lukecwik opened a new pull request #12258: URL: https://github.com/apache/beam/pull/12258 This mirrors the logic within Java's OffsetRangeTracker implementation except for the subtraction overflow handling. I tried fixing one thing but ran into these other cascading issues: * Update

[GitHub] [beam] pabloem commented on pull request #12236: [BEAM-10419] Ignore missing temporary files in retrys.

pabloem commented on pull request #12236: URL: https://github.com/apache/beam/pull/12236#issuecomment-658454027 thanks! @lastomato cool implementaiton : ) This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] pabloem merged pull request #12236: [BEAM-10419] Ignore missing temporary files in retrys.

pabloem merged pull request #12236: URL: https://github.com/apache/beam/pull/12236 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] aaltay commented on a change in pull request #12252: [BEAM-7390] Add combineglobally code snippets

aaltay commented on a change in pull request #12252: URL: https://github.com/apache/beam/pull/12252#discussion_r454689609 ## File path: website/www/site/content/en/documentation/transforms/python/aggregation/combineglobally.md ## @@ -14,29 +14,197 @@ WITHOUT WARRANTIES OR

[GitHub] [beam] amaliujia commented on pull request #11975: [BEAM-9198] BeamSQL aggregation analytics functionality

amaliujia commented on pull request #11975: URL: https://github.com/apache/beam/pull/11975#issuecomment-658451009 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] saavannanavati commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit

saavannanavati commented on pull request #12257: URL: https://github.com/apache/beam/pull/12257#issuecomment-658449426 An estimate for the extra time this will add to `py38-cloud`. These numbers are from testing on my machine. - Without coverage, it runs in ~ `9m 40s` - With

[GitHub] [beam] aaltay commented on a change in pull request #12249: Make tracking/cleanup of cache and in-environment states per pipeline

aaltay commented on a change in pull request #12249: URL: https://github.com/apache/beam/pull/12249#discussion_r454687033 ## File path: sdks/python/apache_beam/runners/interactive/pipeline_instrument.py ## @@ -556,19 +556,26 @@ def _process(self, pcoll): if not

[GitHub] [beam] saavannanavati commented on pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit

saavannanavati commented on pull request #12257: URL: https://github.com/apache/beam/pull/12257#issuecomment-658447736 R: @udim R: @robertwb CC: @tysonjh This is an automated message from the Apache Git Service.

[GitHub] [beam] saavannanavati opened a new pull request #12257: [BEAM-2762] Generate Python coverage reports during pre-commit

saavannanavati opened a new pull request #12257: URL: https://github.com/apache/beam/pull/12257 This PR enables the generation of Python coverage reports during pre-commit in the `py38-cloud` Tox environment. - The resultant reports are uploaded to `codecov.io`, as can be seen

[GitHub] [beam] chamikaramj commented on pull request #12196: [release-2.23.0][BEAM-10308] Make component ID assignments consistent across Pipeline…

chamikaramj commented on pull request #12196: URL: https://github.com/apache/beam/pull/12196#issuecomment-658446507 I think that's fair. If we happen to cut a RC2 we can consider including this. Otherwise we can wait for next release. If we determine that some of our users are hitting

[GitHub] [beam] aaltay commented on pull request #12217: [BEAM-7390] Add cogroupbykey code snippets

aaltay commented on pull request #12217: URL: https://github.com/apache/beam/pull/12217#issuecomment-658445762 Could you update the staged version? I cannot open it. This is an automated message from the Apache Git Service.

[GitHub] [beam] udim merged pull request #12230: [BEAM-7014] filesystemio.py PipeStream rewind fix

udim merged pull request #12230: URL: https://github.com/apache/beam/pull/12230 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [beam] tvalentyn commented on pull request #12196: [release-2.23.0][BEAM-10308] Make component ID assignments consistent across Pipeline…

tvalentyn commented on pull request #12196: URL: https://github.com/apache/beam/pull/12196#issuecomment-658441886 > To be honest I think this situation is somewhat of a gap in the guidance there. There may be cases where its worth delaying a release for a severe bugfix even if it's a

[GitHub] [beam] jaketf commented on pull request #12254: [BEAM-XXXX] Add note on zetaSQL dependencies

jaketf commented on pull request #12254: URL: https://github.com/apache/beam/pull/12254#issuecomment-658436439 R: @kennknowles This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] ibzib opened a new pull request #12256: [BEAM-10487] Link release blog posts on the Downloads page

ibzib opened a new pull request #12256: URL: https://github.com/apache/beam/pull/12256 R: @tvalentyn Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose

[GitHub] [beam] allenpradeep opened a new pull request #12255: [BEAM-9910]Add mutation byte counters to SpannerIO.write

allenpradeep opened a new pull request #12255: URL: https://github.com/apache/beam/pull/12255 Adding mutation byte counters for all mutations written to Spanner. This will help monitor job progress. @nielm @chamikaramj Thank you for your

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454663699 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ExpressionConverter.java ## @@

[GitHub] [beam] pabloem commented on pull request #12236: [BEAM-10419] Ignore missing temporary files in retrys.

pabloem commented on pull request #12236: URL: https://github.com/apache/beam/pull/12236#issuecomment-658426505 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] kennknowles merged pull request #11792: [BEAM-10435] Add ValidatesRunner task for local_job_service and Java SDK harness

kennknowles merged pull request #11792: URL: https://github.com/apache/beam/pull/11792 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] kennknowles commented on pull request #11792: [BEAM-10435] Add ValidatesRunner task for local_job_service and Java SDK harness

kennknowles commented on pull request #11792: URL: https://github.com/apache/beam/pull/11792#issuecomment-658426050 OK this is known flake https://issues.apache.org/jira/browse/BEAM-10470 in `JdbcDriverTest`, totally unrelated to this and shouldn't even be running. At this point I feel

[GitHub] [beam] pabloem commented on pull request #12236: [BEAM-10419] Ignore missing temporary files in retrys.

pabloem commented on pull request #12236: URL: https://github.com/apache/beam/pull/12236#issuecomment-658425664 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] kennknowles edited a comment on pull request #11639: [BEAM-4440] Throw exception when file to stage is not found, instead of logging a warning

kennknowles edited a comment on pull request #11639: URL: https://github.com/apache/beam/pull/11639#issuecomment-658424957 Some new and confusing flakes in `StreamingDataflowWorkerTest`. Spending too much time triaging and filing these. Here is the build:

[GitHub] [beam] kennknowles commented on pull request #11639: [BEAM-4440] Throw exception when file to stage is not found, instead of logging a warning

kennknowles commented on pull request #11639: URL: https://github.com/apache/beam/pull/11639#issuecomment-658424957 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [beam] jaketf opened a new pull request #12254: [BEAM-XXXX] Add note on zetaSQL dependencies

jaketf opened a new pull request #12254: URL: https://github.com/apache/beam/pull/12254 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454653663 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BeamBigQuerySqlDialect.java ## @@

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454653552 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlTypesUtils.java ## @@ -0,0 +1,42 @@

[GitHub] [beam] jhnmora000 commented on pull request #11975: [BEAM-9198] BeamSQL aggregation analytics functionality

jhnmora000 commented on pull request #11975: URL: https://github.com/apache/beam/pull/11975#issuecomment-658417727 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] robertwb commented on pull request #11974: [BEAM-9547] Add more methods to deferred dataframes.

robertwb commented on pull request #11974: URL: https://github.com/apache/beam/pull/11974#issuecomment-658415560 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] lukecwik merged pull request #12241: [BEAM-10420] Fix minor race condition related to splitting that will cause None has no method 'yyy'

lukecwik merged pull request #12241: URL: https://github.com/apache/beam/pull/12241 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] apilloud commented on pull request #12240: [BEAM-10462] transforms handle Double.NaN

apilloud commented on pull request #12240: URL: https://github.com/apache/beam/pull/12240#issuecomment-658400669 Yes, all tests pass. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] kennknowles merged pull request #12224: [BEAM-10080] Fixes PipelineOptionsFactoryTest and ReflectHelpersTest for Java 11

kennknowles merged pull request #12224: URL: https://github.com/apache/beam/pull/12224 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] apilloud commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

apilloud commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454615048 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BeamBigQuerySqlDialect.java ## @@ -156,7

[GitHub] [beam] KevinGG commented on pull request #12249: Make tracking/cleanup of cache and in-environment states per pipeline

KevinGG commented on pull request #12249: URL: https://github.com/apache/beam/pull/12249#issuecomment-658394934 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] lukecwik commented on pull request #12241: [BEAM-10420] Fix minor race condition related to splitting that will cause None has no method 'yyy'

lukecwik commented on pull request #12241: URL: https://github.com/apache/beam/pull/12241#issuecomment-658393808 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] ihji commented on pull request #11638: [BEAM-9449] Pass PipelineOptions through expansion service

ihji commented on pull request #11638: URL: https://github.com/apache/beam/pull/11638#issuecomment-658391509 Is this still valid? This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] ibzib edited a comment on pull request #12250: [BEAM-10460] Limit SparkPortableExecutionTest to a single thread.

ibzib edited a comment on pull request #12250: URL: https://github.com/apache/beam/pull/12250#issuecomment-658386161 > Can the jobs share a cluster concurrently? For a normal cluster, yes. But I'm not sure if there's a way to share a "local" (embedded) cluster.

[GitHub] [beam] ibzib commented on pull request #12250: [BEAM-10460] Limit SparkPortableExecutionTest to a single thread.

ibzib commented on pull request #12250: URL: https://github.com/apache/beam/pull/12250#issuecomment-658386161 > Can the jobs share a cluster concurrently? For a normal cluster, yes. But I'm not sure if there's a way to do this with a "local" (embedded) cluster.

[GitHub] [beam] kennknowles commented on pull request #11792: [BEAM-10435] Add ValidatesRunner task for local_job_service and Java SDK harness

kennknowles commented on pull request #11792: URL: https://github.com/apache/beam/pull/11792#issuecomment-658385268 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [beam] kennknowles commented on pull request #11792: [BEAM-10435] Add ValidatesRunner task for local_job_service and Java SDK harness

kennknowles commented on pull request #11792: URL: https://github.com/apache/beam/pull/11792#issuecomment-658384672 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [beam] kennknowles commented on pull request #11792: [BEAM-10435] Add ValidatesRunner task for local_job_service and Java SDK harness

kennknowles commented on pull request #11792: URL: https://github.com/apache/beam/pull/11792#issuecomment-658384859 (it could have been interrupted, I didn't dig too deep) This is an automated message from the Apache Git

[GitHub] [beam] kennknowles commented on pull request #11639: [BEAM-4440] Throw exception when file to stage is not found, instead of logging a warning

kennknowles commented on pull request #11639: URL: https://github.com/apache/beam/pull/11639#issuecomment-658383999 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [beam] kennknowles commented on pull request #12250: [BEAM-10460] Limit SparkPortableExecutionTest to a single thread.

kennknowles commented on pull request #12250: URL: https://github.com/apache/beam/pull/12250#issuecomment-658383654 Might want to confirm deflaking. Merging despite my question because if it helps that's great. This is an

[GitHub] [beam] kennknowles merged pull request #12250: [BEAM-10460] Limit SparkPortableExecutionTest to a single thread.

kennknowles merged pull request #12250: URL: https://github.com/apache/beam/pull/12250 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] kennknowles commented on pull request #12250: [BEAM-10460] Limit SparkPortableExecutionTest to a single thread.

kennknowles commented on pull request #12250: URL: https://github.com/apache/beam/pull/12250#issuecomment-658382411 Can the jobs share a cluster concurrently? This is an automated message from the Apache Git Service. To

[GitHub] [beam] kennknowles closed pull request #12243: [BEAM-10460] Increase SparkPortableExecutionTest.testExecution timeout to 10 minutes

kennknowles closed pull request #12243: URL: https://github.com/apache/beam/pull/12243 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] lastomato commented on pull request #12236: [BEAM-10419] Ignore missing temporary files in retrys.

lastomato commented on pull request #12236: URL: https://github.com/apache/beam/pull/12236#issuecomment-658381151 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] kennknowles commented on pull request #4341: [BEAM-3354] Fixed a bug that prevented processing time timers to be reset

kennknowles commented on pull request #4341: URL: https://github.com/apache/beam/pull/4341#issuecomment-658376373 @ffDante can you file a Jira describing what you have encountered? https://issues.apache.org/jira/secure/CreateIssue!default.jspa

[GitHub] [beam] kennknowles commented on pull request #12226: [BEAM-10450] [BEAM-10449] Fix empty flatten bundle processing.

kennknowles commented on pull request #12226: URL: https://github.com/apache/beam/pull/12226#issuecomment-658375602 Please add additional failing runs to the ticket so that we can notice them. This is an automated message

[GitHub] [beam] kennknowles commented on pull request #12226: [BEAM-10450] [BEAM-10449] Fix empty flatten bundle processing.

kennknowles commented on pull request #12226: URL: https://github.com/apache/beam/pull/12226#issuecomment-658375455 The `SparkPortableExecutionTest` failure is tracked at https://issues.apache.org/jira/browse/BEAM-10460. Not yet known whether it is a timeout flake or a failure flake.

[GitHub] [beam] kennknowles merged pull request #12253: Exclude beam.* from license download since these are invalid artifact coordinates

kennknowles merged pull request #12253: URL: https://github.com/apache/beam/pull/12253 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] KevinGG commented on pull request #12249: Make tracking/cleanup of cache and in-environment states per pipeline

KevinGG commented on pull request #12249: URL: https://github.com/apache/beam/pull/12249#issuecomment-658371339 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] KevinGG commented on pull request #12249: Make tracking/cleanup of cache and in-environment states per pipeline

KevinGG commented on pull request #12249: URL: https://github.com/apache/beam/pull/12249#issuecomment-658371441 Run Portable_Python PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] amaliujia commented on pull request #12240: [BEAM-10462] transforms handle Double.NaN

amaliujia commented on pull request #12240: URL: https://github.com/apache/beam/pull/12240#issuecomment-658370577 Have you run beamimport to test this PR internally? This is an automated message from the Apache Git Service.

[GitHub] [beam] kennknowles commented on pull request #12253: Exclude beam.* from license download since these are invalid artifact coordinates

kennknowles commented on pull request #12253: URL: https://github.com/apache/beam/pull/12253#issuecomment-658359606 Noting that `:sdks:java:container:pullLicenses` has succeeded, indicating the problem is solved. This is an

[GitHub] [beam] kennknowles commented on pull request #12202: [BEAM-10407,10408] Schema Capable IO Table Provider Wrappers

kennknowles commented on pull request #12202: URL: https://github.com/apache/beam/pull/12202#issuecomment-658358581 I think Luke said the same thing. You put `TableProvider` interface somewhere everyone can depend on. I believe in your proposal it might end up in up in java core. I don't

[GitHub] [beam] kennknowles commented on pull request #12202: [BEAM-10407,10408] Schema Capable IO Table Provider Wrappers

kennknowles commented on pull request #12202: URL: https://github.com/apache/beam/pull/12202#issuecomment-658357325 Didn't have time to look too closely, but once `TableProvider` moved to a service loader model, SQL should not have a `provided` dependency on the specific `TableProvider`

[GitHub] [beam] robertwb commented on pull request #12248: Cleanup some old pipeline examples.

robertwb commented on pull request #12248: URL: https://github.com/apache/beam/pull/12248#issuecomment-658354934 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] robertwb commented on pull request #12248: Cleanup some old pipeline examples.

robertwb commented on pull request #12248: URL: https://github.com/apache/beam/pull/12248#issuecomment-658354699 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] kennknowles commented on pull request #12253: Exclude beam.* from license download since these are invalid artifact coordinates

kennknowles commented on pull request #12253: URL: https://github.com/apache/beam/pull/12253#issuecomment-658352331 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] kennknowles opened a new pull request #12253: Exclude beam.* from license download since these are invalid artifact coordinates

kennknowles opened a new pull request #12253: URL: https://github.com/apache/beam/pull/12253 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [x] [**Choose

[GitHub] [beam] amaliujia closed pull request #12092: [DO NOT MERGE] Prototype TPCDS Benchmark for BeamSQL

amaliujia closed pull request #12092: URL: https://github.com/apache/beam/pull/12092 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] apilloud commented on pull request #12240: [BEAM-10462] transforms handle Double.NaN

apilloud commented on pull request #12240: URL: https://github.com/apache/beam/pull/12240#issuecomment-658339459 R: @amaliujia This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] Ardagan commented on pull request #11582: [BEAM-9650] Add ReadAllFromBigQuery PTransform

Ardagan commented on pull request #11582: URL: https://github.com/apache/beam/pull/11582#issuecomment-658338739 Run Python 2 PostCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] robertwb merged pull request #12246: Add Python snippets in testing guide.

robertwb merged pull request #12246: URL: https://github.com/apache/beam/pull/12246 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r45413 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlTypesUtils.java ## @@ -0,0 +1,35 @@

[GitHub] [beam] ZijieSong946 commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

ZijieSong946 commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454554801 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ExpressionConverter.java ## @@

[GitHub] [beam] lastomato commented on pull request #12236: [BEAM-10419] Ignore missing temporary files in retrys.

lastomato commented on pull request #12236: URL: https://github.com/apache/beam/pull/12236#issuecomment-658335667 This is ready for review. The error can be reproduced by throwing an error (not caught) in `ImportFn`, the retrys will fail since we always remove the temporary files.

[GitHub] [beam] KevinGG commented on a change in pull request #12249: Make tracking/cleanup of cache and in-environment states per pipeline

KevinGG commented on a change in pull request #12249: URL: https://github.com/apache/beam/pull/12249#discussion_r454545613 ## File path: sdks/python/apache_beam/runners/interactive/pipeline_instrument.py ## @@ -556,19 +556,26 @@ def _process(self, pcoll): if not

[GitHub] [beam] KevinGG commented on a change in pull request #12249: Make tracking/cleanup of cache and in-environment states per pipeline

KevinGG commented on a change in pull request #12249: URL: https://github.com/apache/beam/pull/12249#discussion_r454543220 ## File path: sdks/python/apache_beam/runners/interactive/interactive_runner.py ## @@ -259,7 +247,8 @@ def read(self, pcoll, include_window_info=False):

[GitHub] [beam] robinyqiu commented on a change in pull request #12174: [BEAM-10239] Support ZetaSQL NUMERIC type in BeamSQL

robinyqiu commented on a change in pull request #12174: URL: https://github.com/apache/beam/pull/12174#discussion_r454526705 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/ExpressionConverter.java ## @@

  1   2   >