Re: [PR] Bump github.com/docker/docker from 25.0.5+incompatible to 26.1.1+incompatible in /sdks [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31144: URL: https://github.com/apache/beam/pull/31144#issuecomment-2087983574 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

[PR] Bump github.com/docker/docker from 25.0.5+incompatible to 26.1.1+incompatible in /sdks [beam]

2024-04-30 Thread via GitHub
dependabot[bot] opened a new pull request, #31144: URL: https://github.com/apache/beam/pull/31144 Bumps [github.com/docker/docker](https://github.com/docker/docker) from 25.0.5+incompatible to 26.1.1+incompatible. Release notes Sourced from

Re: [PR] Bump github.com/docker/docker from 25.0.5+incompatible to 26.1.0+incompatible in /sdks [beam]

2024-04-30 Thread via GitHub
dependabot[bot] commented on PR #31077: URL: https://github.com/apache/beam/pull/31077#issuecomment-2087956972 Superseded by #31144. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Bump github.com/docker/docker from 25.0.5+incompatible to 26.1.0+incompatible in /sdks [beam]

2024-04-30 Thread via GitHub
dependabot[bot] closed pull request #31077: Bump github.com/docker/docker from 25.0.5+incompatible to 26.1.0+incompatible in /sdks URL: https://github.com/apache/beam/pull/31077 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
m-trieu commented on code in PR #31133: URL: https://github.com/apache/beam/pull/31133#discussion_r1585844460 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java: ## @@ -387,24 +393,53 @@ private

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
m-trieu commented on code in PR #31133: URL: https://github.com/apache/beam/pull/31133#discussion_r1585833306 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java: ## @@ -387,24 +393,53 @@ private

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
m-trieu commented on code in PR #31133: URL: https://github.com/apache/beam/pull/31133#discussion_r1585824700 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java: ## @@ -387,24 +393,53 @@ private

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
m-trieu commented on code in PR #31133: URL: https://github.com/apache/beam/pull/31133#discussion_r1585823786 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java: ## @@ -387,24 +393,53 @@ private

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
m-trieu commented on code in PR #31133: URL: https://github.com/apache/beam/pull/31133#discussion_r1585822456 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/streaming/config/StreamingPipelineConfig.java: ## @@ -0,0 +1,73 @@

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
m-trieu commented on code in PR #31133: URL: https://github.com/apache/beam/pull/31133#discussion_r1585821365 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java: ## @@ -387,24 +393,53 @@ private

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
m-trieu commented on code in PR #31133: URL: https://github.com/apache/beam/pull/31133#discussion_r1585820746 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java: ## @@ -387,24 +393,53 @@ private

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
m-trieu commented on code in PR #31133: URL: https://github.com/apache/beam/pull/31133#discussion_r1585820350 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java: ## @@ -373,10 +350,39 @@ private

Re: [PR] Revert addition of Redistribute and tests to Java SDK, and translations from runners [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31140: URL: https://github.com/apache/beam/pull/31140#issuecomment-2087839485 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

Re: [PR] [Python][Pipeline Options] Recursively Get All Subclasses [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31141: URL: https://github.com/apache/beam/pull/31141#issuecomment-2087839420 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

Re: [PR] Move all elasticsearch testing to elasticsearch testing job [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31143: URL: https://github.com/apache/beam/pull/31143#issuecomment-2087839319 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @Abacn for label build. Available commands: - `stop

Re: [PR] Move all elasticsearch testing to elasticsearch testing job [beam]

2024-04-30 Thread via GitHub
Abacn commented on code in PR #31143: URL: https://github.com/apache/beam/pull/31143#discussion_r1585759160 ## .github/workflows/beam_PreCommit_Java_ElasticSearch_IO_Direct.yml: ## @@ -22,12 +22,16 @@ on: paths: - "sdks/java/io/elasticsearch/**" -

Re: [PR] Move all elasticsearch testing to elasticsearch testing job [beam]

2024-04-30 Thread via GitHub
Abacn commented on code in PR #31143: URL: https://github.com/apache/beam/pull/31143#discussion_r1585763975 ## .github/workflows/beam_PreCommit_Java_ElasticSearch_IO_Direct.yml: ## @@ -22,12 +22,16 @@ on: paths: - "sdks/java/io/elasticsearch/**" -

Re: [PR] Revert addition of Redistribute and tests to Java SDK, and translations from runners [beam]

2024-04-30 Thread via GitHub
kennknowles commented on PR #31140: URL: https://github.com/apache/beam/pull/31140#issuecomment-2087792928 Ah I see `Artifact download URL: https://github.com/apache/beam/actions/runs/8902890984/artifacts/1462494697` -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Move all elasticsearch testing to elasticsearch testing job [beam]

2024-04-30 Thread via GitHub
Abacn commented on code in PR #31143: URL: https://github.com/apache/beam/pull/31143#discussion_r1585759160 ## .github/workflows/beam_PreCommit_Java_ElasticSearch_IO_Direct.yml: ## @@ -22,12 +22,16 @@ on: paths: - "sdks/java/io/elasticsearch/**" -

Re: [PR] Revert addition of Redistribute and tests to Java SDK, and translations from runners [beam]

2024-04-30 Thread via GitHub
kennknowles commented on PR #31140: URL: https://github.com/apache/beam/pull/31140#issuecomment-2087792285 How did you find the warning through the GHA interface (or Gradle Scan) ? I have always failed to find it and had to repro locally -- This is an automated message from the Apache

Re: [PR] Revert addition of Redistribute and tests to Java SDK, and translations from runners [beam]

2024-04-30 Thread via GitHub
kennknowles commented on PR #31140: URL: https://github.com/apache/beam/pull/31140#issuecomment-2087790690 Yea looks like a real issue with KafkaIO but not changed by this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [Draft] Return methods in a lexicographical order from ReflectUtils.getMethods() [beam]

2024-04-30 Thread via GitHub
kennknowles commented on PR #30279: URL: https://github.com/apache/beam/pull/30279#issuecomment-2087789292 Excellent. I was worried this would cause update incompatibility just as you described. But it sounds like it is fine.  -- This is an automated message from the Apache Git

Re: [PR] Revert addition of Redistribute and tests to Java SDK, and translations from runners [beam]

2024-04-30 Thread via GitHub
Abacn commented on PR #31140: URL: https://github.com/apache/beam/pull/31140#issuecomment-2087787286 ':sdks:java:io:kafka:spotbugsMain' failing on PR but I checkout the PR branch it passed locally. Seems to be unknown test infrastructure issue -- This is an automated message from the

Re: [I] [Bug]: .xs() method on DeferredSeries and DeferredDataFrame doesn't work right when returning single items [beam]

2024-04-30 Thread via GitHub
tvalentyn commented on issue #28559: URL: https://github.com/apache/beam/issues/28559#issuecomment-2087785335 Seeing a similar error on a different operation: ``` from apache_beam.dataframe.doctests import teststring test = """ With a DataFrame >>>

Re: [PR] Move all elasticsearch testing to elasticsearch testing job [beam]

2024-04-30 Thread via GitHub
kennknowles commented on code in PR #31143: URL: https://github.com/apache/beam/pull/31143#discussion_r1585749416 ## .github/workflows/beam_PreCommit_Java_ElasticSearch_IO_Direct.yml: ## @@ -22,12 +22,16 @@ on: paths: - "sdks/java/io/elasticsearch/**" -

[PR] Move all elasticsearch testing to elasticsearch testing job [beam]

2024-04-30 Thread via GitHub
kennknowles opened a new pull request, #31143: URL: https://github.com/apache/beam/pull/31143 I noticed that elasticsearch tests flaked in the "PreCommit Java IOs Direct / beam_PreCommit_Java_IOs_Direct" job and intended to separate them to isolate that flaky signal. In fact, there

Re: [PR] Set backlog in gauge metric for Kafka Reader [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31137: URL: https://github.com/apache/beam/pull/31137#issuecomment-2087759947 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

Re: [PR] Terraform module to provision a kafka proxy [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31142: URL: https://github.com/apache/beam/pull/31142#issuecomment-2087759864 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @shunping added as fallback since no labels match configuration

[PR] Terraform module to provision a kafka proxy [beam]

2024-04-30 Thread via GitHub
damondouglas opened a new pull request, #31142: URL: https://github.com/apache/beam/pull/31142 This PR provides a terraform module to provision a compute engine instance bastion host to act as a proxy to a private Kafka cluster. See README committed in this PR for description,

Re: [PR] Add support for int gauge metrics in DataflowRunner [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31136: URL: https://github.com/apache/beam/pull/31136#issuecomment-2087717605 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

[PR] [Pipeline Options] Recursively Get All Subclasses [beam]

2024-04-30 Thread via GitHub
manav-glean opened a new pull request, #31141: URL: https://github.com/apache/beam/pull/31141 **Please** add a meaningful description for your change here Fixes #31139 by changing the subclasses we iterate to be recursively getting all subclasses of PipelineOptions.

[PR] Revert addition of Redistribute and tests to Java SDK, and translations from runners [beam]

2024-04-30 Thread via GitHub
kennknowles opened a new pull request, #31140: URL: https://github.com/apache/beam/pull/31140 This new transform and tests did not cover the portable variations of enough runners. I will roll back and roll forward the composite first, which should pass all ValidatesRunner tests on all

Re: [PR] Add Redistribute transform to model, Java SDK, and most active runners [beam]

2024-04-30 Thread via GitHub
kennknowles commented on PR #30545: URL: https://github.com/apache/beam/pull/30545#issuecomment-2087706789 I am opening a revert in a minute but taking a look also. - Samza PVR is existing issue with Reshuffle so sickbay is parity - ULR seems to have some completely wild

Re: [PR] Update game_stats.py [beam]

2024-04-30 Thread via GitHub
tvalentyn merged PR #31134: URL: https://github.com/apache/beam/pull/31134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Update game_stats.py [beam]

2024-04-30 Thread via GitHub
tvalentyn commented on PR #31134: URL: https://github.com/apache/beam/pull/31134#issuecomment-2087600724 Thanks. Filed https://github.com/apache/beam/issues/31138 for the failing test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[PR] Add support for int gauge metrics in DataflowRunner [beam]

2024-04-30 Thread via GitHub
Naireen opened a new pull request, #31136: URL: https://github.com/apache/beam/pull/31136 Add support for int gauge metrics in DataflowRunner Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and

[PR] Set backlog in gauge metric for Kafka Reader [beam]

2024-04-30 Thread via GitHub
Naireen opened a new pull request, #31137: URL: https://github.com/apache/beam/pull/31137 There are gauges created for per partition backlog in the kafka reader, but its never actually set. Thank you for your contribution! Follow this checklist to help

Re: [PR] FnAPI proto changes for ordered list state. [beam]

2024-04-30 Thread via GitHub
robertwb commented on code in PR #31092: URL: https://github.com/apache/beam/pull/31092#discussion_r1585535722 ## model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto: ## @@ -1063,6 +1087,10 @@ message StateAppendRequest { // Represents a

Re: [PR] FnAPI proto changes for ordered list state. [beam]

2024-04-30 Thread via GitHub
robertwb commented on code in PR #31092: URL: https://github.com/apache/beam/pull/31092#discussion_r1585535722 ## model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto: ## @@ -1063,6 +1087,10 @@ message StateAppendRequest { // Represents a

Re: [PR] Update game_stats.py [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31134: URL: https://github.com/apache/beam/pull/31134#issuecomment-2087322639 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @tvalentyn for label python. Available commands: -

Re: [PR] Add `.python-version` to .gitignore [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31135: URL: https://github.com/apache/beam/pull/31135#issuecomment-2087322358 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damccorm for label build. Available commands: - `stop

Re: [PR] Optimise View.asList() side inputs for iterating rather than for indexing. [beam]

2024-04-30 Thread via GitHub
robertwb merged PR #31087: URL: https://github.com/apache/beam/pull/31087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Add `.python-version` to .gitignore [beam]

2024-04-30 Thread via GitHub
shahar1 opened a new pull request, #31135: URL: https://github.com/apache/beam/pull/31135 As `pyenv` is recommended for use in [beam's Python's Developer Guide](https://cwiki.apache.org/confluence/display/BEAM/), it makes sense to include `.python-version` in the `.gitignore` file. It

[PR] Update game_stats.py [beam]

2024-04-30 Thread via GitHub
aaltay opened a new pull request, #31134: URL: https://github.com/apache/beam/pull/31134 Fixing a typo in examples & docs. **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help

Re: [PR] Suppress BigQuery read stream splitAtFraction when API busy call or timeout [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31125: URL: https://github.com/apache/beam/pull/31125#issuecomment-2086989569 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git

Re: [PR] Suppress BigQuery read stream splitAtFraction when API busy call or timeout [beam]

2024-04-30 Thread via GitHub
Abacn commented on PR #31125: URL: https://github.com/apache/beam/pull/31125#issuecomment-2086981568 R: @ahmedabu98 (since you reviewed #31096) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Suppress BigQuery read stream splitAtFraction when API busy call or timeout [beam]

2024-04-30 Thread via GitHub
Abacn commented on code in PR #31125: URL: https://github.com/apache/beam/pull/31125#discussion_r1581648996 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryStorageStreamSource.java: ## @@ -388,8 +398,22 @@ public synchronized

Re: [PR] Add Redistribute transform to model, Java SDK, and most active runners [beam]

2024-04-30 Thread via GitHub
kennknowles commented on PR #30545: URL: https://github.com/apache/beam/pull/30545#issuecomment-2086869589 Ah, I neglected the PVR suites. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Enable BigQueryMetrics by default [beam]

2024-04-30 Thread via GitHub
Abacn commented on PR #31015: URL: https://github.com/apache/beam/pull/31015#issuecomment-2086841035 @JayajP merged, would you mind open a new PR to announce (promote) this change in CHANGES.md ? -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Enable BigQueryMetrics by default [beam]

2024-04-30 Thread via GitHub
Abacn merged PR #31015: URL: https://github.com/apache/beam/pull/31015 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Fix typo [beam]

2024-04-30 Thread via GitHub
Abacn merged PR #31063: URL: https://github.com/apache/beam/pull/31063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Elasticsearch enable Point In Time based searches [beam]

2024-04-30 Thread via GitHub
Abacn merged PR #30824: URL: https://github.com/apache/beam/pull/30824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add Redistribute transform to model, Java SDK, and most active runners [beam]

2024-04-30 Thread via GitHub
Abacn commented on PR #30545: URL: https://github.com/apache/beam/pull/30545#issuecomment-2086781686 Looks like several VR PostCommits failing after this PR: https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_ValidatesRunner_ULR.yml?query=event%3Aschedule

Re: [I] [Feature Request]: Set quota project in `beam.io.ReadFromBigQuery` [beam]

2024-04-30 Thread via GitHub
brucearctor commented on issue #31126: URL: https://github.com/apache/beam/issues/31126#issuecomment-2086697350 This should be pretty good --> https://github.com/apache/beam/blob/master/CONTRIBUTING.md If you find a problem [ or that is outdated ], let's overcome and fix the docs

Re: [PR] [Prism] Enable Java validatesRunner tests on Prism [beam]

2024-04-30 Thread via GitHub
damondouglas commented on PR #31075: URL: https://github.com/apache/beam/pull/31075#issuecomment-2086694338 @lostluck This is ready. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] [Feature Request]: Set quota project in `beam.io.ReadFromBigQuery` [beam]

2024-04-30 Thread via GitHub
shahar1 commented on issue #31126: URL: https://github.com/apache/beam/issues/31126#issuecomment-2086670421 .take-issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Elasticsearch enable Point In Time based searches [beam]

2024-04-30 Thread via GitHub
prodriguezdefino commented on code in PR #30824: URL: https://github.com/apache/beam/pull/30824#discussion_r1585325343 ## sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java: ## @@ -832,15 +857,17 @@ public Read withMetadata() {

Re: [PR] FnAPI proto changes for ordered list state. [beam]

2024-04-30 Thread via GitHub
shunping commented on PR #31092: URL: https://github.com/apache/beam/pull/31092#issuecomment-2086033582 All other failed tests are related to macos, which is an issue related to using the latest macos artifact. @damccorm already addressed this in #31115 and #31123 by pinning an older

Re: [I] [Feature Request]: Set quota project in `beam.io.ReadFromBigQuery` [beam]

2024-04-30 Thread via GitHub
shahar1 commented on issue #31126: URL: https://github.com/apache/beam/issues/31126#issuecomment-2086027904 > @shahar1 sounds like you've got a decent idea/design in mind, which could be supported. > > Are you interested in contributing? Feel free to start and include me on PRs,

Re: [PR] FnAPI proto changes for ordered list state. [beam]

2024-04-30 Thread via GitHub
shunping commented on PR #31092: URL: https://github.com/apache/beam/pull/31092#issuecomment-2085929173 Run Java PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Elasticsearch enable Point In Time based searches [beam]

2024-04-30 Thread via GitHub
Abacn commented on code in PR #30824: URL: https://github.com/apache/beam/pull/30824#discussion_r1585079959 ## sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java: ## @@ -832,15 +857,17 @@ public Read withMetadata() { /**

Re: [PR] remove atomic and wait from strimzi helm terrafrom [beam]

2024-04-30 Thread via GitHub
damondouglas merged PR #31132: URL: https://github.com/apache/beam/pull/31132 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] remove atomic and wait from strimzi helm terrafrom [beam]

2024-04-30 Thread via GitHub
damondouglas commented on PR #31132: URL: https://github.com/apache/beam/pull/31132#issuecomment-2085677410 Thank you for doing this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] [Feature Request]: Set quota project in `beam.io.ReadFromBigQuery` [beam]

2024-04-30 Thread via GitHub
brucearctor commented on issue #31126: URL: https://github.com/apache/beam/issues/31126#issuecomment-2085660634 @shahar1 sounds like you've got a decent idea/design in mind, which could be supported. Are you interested in contributing? Feel free to start and include me on PRs, if

Re: [PR] FnAPI proto changes for ordered list state. [beam]

2024-04-30 Thread via GitHub
shunping commented on code in PR #31092: URL: https://github.com/apache/beam/pull/31092#discussion_r1584948151 ## model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto: ## @@ -1063,6 +1087,10 @@ message StateAppendRequest { // Represents a

Re: [I] [Bug]: WriteToFiles in python leave few records in temp directory when writing to large number (100+) of files [beam]

2024-04-30 Thread via GitHub
kennknowles commented on issue #29515: URL: https://github.com/apache/beam/issues/29515#issuecomment-2085536952 Any update here? I suggest downgrade to P2 if we cannot get enough information to reproduce and understand it. -- This is an automated message from the Apache Git Service. To

Re: [PR] FnAPI proto changes for ordered list state. [beam]

2024-04-30 Thread via GitHub
shunping commented on PR #31092: URL: https://github.com/apache/beam/pull/31092#issuecomment-2085530945 Run Java PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] FnAPI proto changes for ordered list state. [beam]

2024-04-30 Thread via GitHub
shunping commented on code in PR #31092: URL: https://github.com/apache/beam/pull/31092#discussion_r1584948151 ## model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto: ## @@ -1063,6 +1087,10 @@ message StateAppendRequest { // Represents a

Re: [PR] Bump com.gradle.enterprise from 3.15.1 to 3.17.2 [beam]

2024-04-30 Thread via GitHub
kennknowles merged PR #31032: URL: https://github.com/apache/beam/pull/31032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] FnAPI proto changes for ordered list state. [beam]

2024-04-30 Thread via GitHub
shunping commented on PR #31092: URL: https://github.com/apache/beam/pull/31092#issuecomment-2085530713 Run GoPortable PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Allow users to configure wait options for new Neo4j databases [beam]

2024-04-30 Thread via GitHub
Abacn merged PR #31129: URL: https://github.com/apache/beam/pull/31129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [Python] Return dicts from enrichment handlers to save conversion cost to rows. [beam]

2024-04-30 Thread via GitHub
riteshghorse closed pull request #30754: [Python] Return dicts from enrichment handlers to save conversion cost to rows. URL: https://github.com/apache/beam/pull/30754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
scwhittle commented on code in PR #31133: URL: https://github.com/apache/beam/pull/31133#discussion_r1584811492 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java: ## @@ -387,24 +393,53 @@ private

Re: [PR] Fix typo [beam]

2024-04-30 Thread via GitHub
shunping commented on PR #31063: URL: https://github.com/apache/beam/pull/31063#issuecomment-2085365887 LGTM! Thanks for the fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] add finalizers to kafka loadbalancer services [beam]

2024-04-30 Thread via GitHub
Abacn merged PR #31130: URL: https://github.com/apache/beam/pull/31130 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [yaml] Add generic expression to filter [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #30086: URL: https://github.com/apache/beam/pull/30086#issuecomment-2085225152 This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This

Re: [PR] [yaml] Add generic expression to filter [beam]

2024-04-30 Thread via GitHub
github-actions[bot] closed pull request #30086: [yaml] Add generic expression to filter URL: https://github.com/apache/beam/pull/30086 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31133: URL: https://github.com/apache/beam/pull/31133#issuecomment-2085157367 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

Re: [PR] add finalizers to kafka loadbalancer services [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31130: URL: https://github.com/apache/beam/pull/31130#issuecomment-2085121644 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git

Re: [PR] add finalizers to kafka loadbalancer services [beam]

2024-04-30 Thread via GitHub
volatilemolotov commented on PR #31130: URL: https://github.com/apache/beam/pull/31130#issuecomment-2085119759 R: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] add computation state cache, config loaders, and status pages [beam]

2024-04-30 Thread via GitHub
m-trieu opened a new pull request, #31133: URL: https://github.com/apache/beam/pull/31133 Add computation state cache and config loaders that load the computation state cache move status pages out of StreamingDataflowWorker file R: @scwhittle

Re: [PR] Allow users to configure wait options for new Neo4j databases [beam]

2024-04-30 Thread via GitHub
fbiville commented on code in PR #31129: URL: https://github.com/apache/beam/pull/31129#discussion_r1584453523 ## it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOptions.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

[PR] remove atomic and wait from strimzi helm terrafrom [beam]

2024-04-30 Thread via GitHub
volatilemolotov opened a new pull request, #31132: URL: https://github.com/apache/beam/pull/31132 removes wait so it does not time out if the cluster needs time to scale. also removes atomic as atomic will set wait to true Thank you for your

Re: [PR] remove atomic and wait from strimzi helm terrafrom [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31132: URL: https://github.com/apache/beam/pull/31132#issuecomment-2084788523 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git

Re: [PR] remove atomic and wait from strimzi helm terrafrom [beam]

2024-04-30 Thread via GitHub
volatilemolotov commented on PR #31132: URL: https://github.com/apache/beam/pull/31132#issuecomment-2084786236 R: @damondouglas -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Allow users to configure wait options for new Neo4j databases [beam]

2024-04-30 Thread via GitHub
fbiville commented on PR #31129: URL: https://github.com/apache/beam/pull/31129#issuecomment-208475 > If this is only meant to be used for Dataflow Template testing, it takes time (next Beam release, 2.57.0) to be effective. Could consider fork the resource manager back to Dataflow

Re: [PR] Allow users to configure wait options for new Neo4j databases [beam]

2024-04-30 Thread via GitHub
fbiville commented on code in PR #31129: URL: https://github.com/apache/beam/pull/31129#discussion_r1584392474 ## it/neo4j/src/main/java/org/apache/beam/it/neo4j/Neo4jResourceManager.java: ## @@ -189,11 +194,26 @@ public String getAdminPassword() { return adminPassword;

Re: [PR] Allow users to configure wait options for new Neo4j databases [beam]

2024-04-30 Thread via GitHub
fbiville commented on code in PR #31129: URL: https://github.com/apache/beam/pull/31129#discussion_r1584390455 ## it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOptions.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] [#30789] Add support for Flink 1.18 [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31062: URL: https://github.com/apache/beam/pull/31062#issuecomment-2084741655 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git

Re: [PR] [#30789] Add support for Flink 1.18 [beam]

2024-04-30 Thread via GitHub
je-ik commented on PR #31062: URL: https://github.com/apache/beam/pull/31062#issuecomment-2084739314 R: @je-ik -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [#30789] Add support for Flink 1.18 [beam]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #31062: URL: https://github.com/apache/beam/pull/31062#issuecomment-2084714189 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @jrmccluskey for label go. R: @Abacn for label build.

Re: [I] [Feature Request]: Set quota project in `beam.io.ReadFromBigQuery` [beam]

2024-04-30 Thread via GitHub
shahar1 commented on issue #31126: URL: https://github.com/apache/beam/issues/31126#issuecomment-2084510959 > @shahar1 this level of customization might make sense. > > Let's explore your specific concern for a moment, I might have others, but imagine worth understanding your