Re: [PR] Bump github.com/testcontainers/testcontainers-go from 0.26.0 to 0.30.0 in /sdks [beam]

github-actions[bot] commented on PR #30901: URL: https://github.com/apache/beam/pull/30901#issuecomment-2044180860 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

[PR] Bump google.golang.org/grpc from 1.62.1 to 1.63.2 in /sdks [beam]

dependabot[bot] opened a new pull request, #30900: URL: https://github.com/apache/beam/pull/30900 Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.62.1 to 1.63.2. Release notes Sourced from https://github.com/grpc/grpc-go/releases;>google.golang.org/grpc's

Re: [PR] Bump github.com/testcontainers/testcontainers-go from 0.26.0 to 0.29.1 in /sdks [beam]

dependabot[bot] closed pull request #30557: Bump github.com/testcontainers/testcontainers-go from 0.26.0 to 0.29.1 in /sdks URL: https://github.com/apache/beam/pull/30557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[I] The PostCommit XVR JavaUsingPython Dataflow job is flaky [beam]

github-actions[bot] opened a new issue, #30899: URL: https://github.com/apache/beam/issues/30899 The PostCommit XVR JavaUsingPython Dataflow is failing over 50% of the time Please visit

Re: [PR] Bump GCP-BOM to 26.36.0 [beam]

Abacn merged PR #30868: URL: https://github.com/apache/beam/pull/30868 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Bump github.com/testcontainers/testcontainers-go from 0.26.0 to 0.30.0 in /sdks [beam]

dependabot[bot] opened a new pull request, #30901: URL: https://github.com/apache/beam/pull/30901 Bumps [github.com/testcontainers/testcontainers-go](https://github.com/testcontainers/testcontainers-go) from 0.26.0 to 0.30.0. Release notes Sourced from

Re: [PR] Bump github.com/testcontainers/testcontainers-go from 0.26.0 to 0.29.1 in /sdks [beam]

dependabot[bot] commented on PR #30557: URL: https://github.com/apache/beam/pull/30557#issuecomment-2044149633 Superseded by #30901. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Patch release website changes [beam]

damccorm commented on PR #30839: URL: https://github.com/apache/beam/pull/30839#issuecomment-2042721488 R: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Patch release website changes [beam]

github-actions[bot] commented on PR #30839: URL: https://github.com/apache/beam/pull/30839#issuecomment-2042724168 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git

Re: [PR] Patch release website changes [beam]

damccorm commented on PR #30839: URL: https://github.com/apache/beam/pull/30839#issuecomment-2042741503 R: @liferoad since I think Yi is out today -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] [Bug]: Beam Sql is ignoring aliases fields in some situations which causes to huge data loss [beam]

brachipa commented on issue #30498: URL: https://github.com/apache/beam/issues/30498#issuecomment-2042822014 I also tried run calcite (same version as beam uses) unit test with the same query and it works fine, with my fix and without my fix. I think it sounds like an issue in beam

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

liferoad commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1556091869 ## CHANGES.md: ## @@ -80,6 +80,7 @@ ## Bugfixes * Fixed locking issue when shutting down inactive bundle processors. Symptoms of this issue include slowness or

[PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py311 [beam]

dependabot[bot] opened a new pull request, #30891: URL: https://github.com/apache/beam/pull/30891 Bumps [pymongo](https://github.com/mongodb/mongo-python-driver) from 4.6.2 to 4.6.3. Changelog Sourced from

Re: [PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py311 [beam]

github-actions[bot] commented on PR #30891: URL: https://github.com/apache/beam/pull/30891#issuecomment-2043302472 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @riteshghorse for label python. Available commands: -

Re: [PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py38 [beam]

github-actions[bot] commented on PR #30890: URL: https://github.com/apache/beam/pull/30890#issuecomment-2043302615 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damccorm for label python. Available commands: - `stop

Re: [PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py39 [beam]

github-actions[bot] commented on PR #30888: URL: https://github.com/apache/beam/pull/30888#issuecomment-2043302869 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @tvalentyn for label python. Available commands: -

Re: [PR] Support FLOAT32 type in Spanner [beam]

github-actions[bot] commented on PR #30893: URL: https://github.com/apache/beam/pull/30893#issuecomment-2043375457 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

Re: [PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py38 [beam]

damccorm closed pull request #30890: Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py38 URL: https://github.com/apache/beam/pull/30890 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py38 [beam]

dependabot[bot] commented on PR #30890: URL: https://github.com/apache/beam/pull/30890#issuecomment-2043419704 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

jbsabbagh commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1556092498 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java: ## @@ -518,8 +525,11 @@ private ConsumerRecords poll( return

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

jbsabbagh commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1556092498 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java: ## @@ -518,8 +525,11 @@ private ConsumerRecords poll( return

[PR] Support FLOAT32 type in Spanner [beam]

arawind opened a new pull request, #30893: URL: https://github.com/apache/beam/pull/30893 Adds support for the newly added FLOAT32 type in Google Cloud Spanner. Since FLOAT was already mapped to FLOAT64, this change modifies the existing mapping to FLOAT -> FLOAT32 instead.

Re: [I] [Bug]: Beam Sql is ignoring aliases fields in some situations which causes to huge data loss [beam]

kennknowles commented on issue #30498: URL: https://github.com/apache/beam/issues/30498#issuecomment-2043220798 Wow nice detective work. If we still see it in Beam but not Calcite then it must be one of our optimization rules. I would look at the debug trace for when it goes wrong. Or with

Re: [PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py310 [beam]

github-actions[bot] commented on PR #30889: URL: https://github.com/apache/beam/pull/30889#issuecomment-2043302750 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @shunping for label python. Available commands: - `stop

Re: [PR] Add PubSubIO Stress test [beam]

github-actions[bot] commented on PR #30886: URL: https://github.com/apache/beam/pull/30886#issuecomment-2043317594 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git

Re: [PR] Initial Iceberg Sink [beam]

kennknowles commented on code in PR #30797: URL: https://github.com/apache/beam/pull/30797#discussion_r1552651757 ## sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/IcebergIO.java: ## @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Optimize histograms and metric names in BigQuerySinkMetrics [beam]

codecov-commenter commented on PR #30796: URL: https://github.com/apache/beam/pull/30796#issuecomment-2043422958 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/30796?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

jbsabbagh commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1556092498 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java: ## @@ -518,8 +525,11 @@ private ConsumerRecords poll( return

[PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py39 [beam]

dependabot[bot] opened a new pull request, #30888: URL: https://github.com/apache/beam/pull/30888 Bumps [pymongo](https://github.com/mongodb/mongo-python-driver) from 4.6.2 to 4.6.3. Changelog Sourced from

[PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py310 [beam]

dependabot[bot] opened a new pull request, #30889: URL: https://github.com/apache/beam/pull/30889 Bumps [pymongo](https://github.com/mongodb/mongo-python-driver) from 4.6.2 to 4.6.3. Changelog Sourced from

[PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py38 [beam]

dependabot[bot] opened a new pull request, #30890: URL: https://github.com/apache/beam/pull/30890 Bumps [pymongo](https://github.com/mongodb/mongo-python-driver) from 4.6.2 to 4.6.3. Changelog Sourced from

Re: [PR] Create YAML Join Transform [beam]

Polber commented on code in PR #30734: URL: https://github.com/apache/beam/pull/30734#discussion_r1556253586 ## sdks/python/apache_beam/yaml/yaml_join_test.py: ## @@ -0,0 +1,210 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

Re: [PR] Duet AI Prompts - Documentation Lookup Without Links [beam]

dariabezkorovaina commented on PR #30873: URL: https://github.com/apache/beam/pull/30873#issuecomment-2043233424 Hi @damccorm, this is the final PR in these Duet AI prompts series: we've created the "no links" versions of all the "documentation lookup" prompts, as well as added some nits

Re: [PR] Duet AI Prompts - Documentation Lookup Without Links [beam]

github-actions[bot] commented on PR #30873: URL: https://github.com/apache/beam/pull/30873#issuecomment-2043255437 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @jrmccluskey added as fallback since no labels match

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

github-actions[bot] commented on PR #30877: URL: https://github.com/apache/beam/pull/30877#issuecomment-2043255243 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @bvolpato for label java. R: @bvolpato for label io.

Re: [PR] Add PubSubIO Stress test [beam]

akashorabek commented on PR #30886: URL: https://github.com/apache/beam/pull/30886#issuecomment-2043315705 R: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py39 [beam]

tvalentyn closed pull request #30888: Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py39 URL: https://github.com/apache/beam/pull/30888 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Bump pymongo from 4.6.2 to 4.6.3 in /sdks/python/container/py39 [beam]

dependabot[bot] commented on PR #30888: URL: https://github.com/apache/beam/pull/30888#issuecomment-2043395144 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] add terraform for utility cluster. Add name override to gke [beam]

volatilemolotov commented on code in PR #30847: URL: https://github.com/apache/beam/pull/30847#discussion_r1555438203 ## .test-infra/terraform/google-cloud-platform/google-kubernetes-engine/cluster.tf: ## @@ -34,7 +34,9 @@ resource "google_container_cluster" "default" {

Re: [PR] add terraform for utility cluster. Add name override to gke [beam]

volatilemolotov commented on code in PR #30847: URL: https://github.com/apache/beam/pull/30847#discussion_r1555483141 ## .test-infra/terraform/google-cloud-platform/google-kubernetes-engine/cluster.tf: ## @@ -34,7 +34,9 @@ resource "google_container_cluster" "default" {

Re: [PR] [Flink] Speed up file write in batch mode by using larger bundle size [beam]

jto commented on PR #30802: URL: https://github.com/apache/beam/pull/30802#issuecomment-2042134252 Sure. I tested it on a job that consumes ~1B records (~150GB). With the Dataset API, runtime is 37min. Passing `--useDataStreamForBatch`, I killed it after 1h+ as it was clearly too

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

xianhualiu commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1556357679 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java: ## @@ -518,8 +525,11 @@ private ConsumerRecords poll( return

Re: [I] [Failing Test]: Python Coverage is failing because of new version of typing-extension release [beam]

riteshghorse closed issue #30806: [Failing Test]: Python Coverage is failing because of new version of typing-extension release URL: https://github.com/apache/beam/issues/30806 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] Revert "Revert #30533: Automatically execute unbounded pipelines in streaming mode." [beam]

damccorm opened a new pull request, #30894: URL: https://github.com/apache/beam/pull/30894 Reverts apache/beam#30706 This change was initially correct -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Windowing Support for the DaskRunner [beam]

cisaacstern commented on PR #27618: URL: https://github.com/apache/beam/pull/27618#issuecomment-2043644883 The necessary upstream fix in Dask was merged!  Once we get a new Dask release that includes this change, I will finish this PR! -- This is an automated message from the

Re: [PR] Initial Iceberg Sink [beam]

chamikaramj commented on code in PR #30797: URL: https://github.com/apache/beam/pull/30797#discussion_r1556356489 ## sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/AppendFilesToTables.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation

Re: [I] [Failing Test]: Python Coverage is failing because of new version of typing-extension release [beam]

riteshghorse commented on issue #30806: URL: https://github.com/apache/beam/issues/30806#issuecomment-2043532181 https://github.com/apache/beam/pull/30863 fixed it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Correct per-entry HashMap overhead in WindmillStateCache [beam]

dmitryor commented on PR #30672: URL: https://github.com/apache/beam/pull/30672#issuecomment-2043539547 I rebased to recent `main` and tests are passing now. I suspect the previous failure was a flake. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

liferoad commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1556372231 ## CHANGES.md: ## @@ -72,6 +72,7 @@ ## Breaking Changes * X behavior was changed ([#X](https://github.com/apache/beam/issues/X)). +* Default consumer polling

Re: [PR] Duet AI Prompts - Documentation Lookup Without Links [beam]

damccorm merged PR #30873: URL: https://github.com/apache/beam/pull/30873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [Bug]: Beam Sql is ignoring aliases fields in some situations which causes to huge data loss [beam]

brachipa commented on issue #30498: URL: https://github.com/apache/beam/issues/30498#issuecomment-2042540303 Ok, I think I find what cause it. calcite checks if expression node is equal to row fields

Re: [PR] Clean doc related to write data in bigquery.py [beam]

github-actions[bot] commented on PR #30887: URL: https://github.com/apache/beam/pull/30887#issuecomment-2042578339 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @liferoad for label python. R: @chamikaramj for label io.

Re: [PR] Implement Web API connector interfaces [beam]

github-actions[bot] commented on PR #30815: URL: https://github.com/apache/beam/pull/30815#issuecomment-2042589105 Reminder, please take a look at this pr: @lostluck @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Implement Web API connector interfaces [beam]

codecov-commenter commented on PR #30815: URL: https://github.com/apache/beam/pull/30815#issuecomment-2042591781 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/30815?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report

Re: [PR] [Typescript] SDK fixes [beam]

github-actions[bot] closed pull request #29889: [Typescript] SDK fixes URL: https://github.com/apache/beam/pull/29889 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [Typescript] SDK fixes [beam]

github-actions[bot] commented on PR #29889: URL: https://github.com/apache/beam/pull/29889#issuecomment-2042643912 This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This

Re: [PR] Bump github.com/aws/smithy-go from 1.20.1 to 1.20.2 in /sdks [beam]

jrmccluskey merged PR #30883: URL: https://github.com/apache/beam/pull/30883 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Initial Iceberg Sink [beam]

chamikaramj commented on code in PR #30797: URL: https://github.com/apache/beam/pull/30797#discussion_r1556446483 ## sdks/java/io/iceberg/src/main/java/org/apache/beam/io/iceberg/RecordWriter.java: ## @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Adding support for high priority queries to xlang transforms writing … [beam]

pabloem commented on PR #30869: URL: https://github.com/apache/beam/pull/30869#issuecomment-2043691501 is this good to go? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] [yaml] backtick generated aliases on sql mappings [beam]

Polber opened a new pull request, #30895: URL: https://github.com/apache/beam/pull/30895 There are cases where `MapToFields` does not work for sql expressions where a field in the incoming schema is a reserved keyword. For example, with an input schema of `{foo: int, timestamp:

Re: [PR] [yaml] backtick generated aliases on sql mappings [beam]

github-actions[bot] commented on PR #30895: URL: https://github.com/apache/beam/pull/30895#issuecomment-2043738129 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git

Re: [PR] [yaml] add use-case example [beam]

liferoad commented on code in PR #30896: URL: https://github.com/apache/beam/pull/30896#discussion_r1556503761 ## sdks/python/apache_beam/yaml/examples/simple_filter_and_combine.yaml: ## @@ -0,0 +1,56 @@ +# coding=utf-8 +# +# Licensed to the Apache Software Foundation (ASF)

Re: [PR] [yaml] add use-case example [beam]

liferoad commented on code in PR #30896: URL: https://github.com/apache/beam/pull/30896#discussion_r1556504460 ## sdks/python/apache_beam/yaml/examples/resources/products.csv: ## @@ -0,0 +1,6 @@ +transaction_id,product_name,category,price +T0012,Headphones,Electronics,59.99

Re: [PR] [yaml] backtick generated aliases on sql mappings [beam]

liferoad commented on code in PR #30895: URL: https://github.com/apache/beam/pull/30895#discussion_r1556512365 ## sdks/python/apache_beam/yaml/yaml_mapping.py: ## @@ -515,7 +515,7 @@ def normalize_fields(pcoll, fields, drop=(), append=False, language='generic'): if

Re: [PR] [yaml] backtick generated aliases on sql mappings [beam]

Polber commented on PR #30895: URL: https://github.com/apache/beam/pull/30895#issuecomment-2043737188 R: @damccorm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [yaml] backtick generated aliases on sql mappings [beam]

Polber commented on PR #30895: URL: https://github.com/apache/beam/pull/30895#issuecomment-2043736968 R: @robertwb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[PR] [yaml] add use-case example [beam]

Polber opened a new pull request, #30896: URL: https://github.com/apache/beam/pull/30896 Adds 2 simple use-case examples to examples catalog. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and

Re: [PR] [yaml] add use-case example [beam]

Polber commented on PR #30896: URL: https://github.com/apache/beam/pull/30896#issuecomment-2043759026 R: @liferoad -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [yaml] add use-case example [beam]

Polber commented on PR #30896: URL: https://github.com/apache/beam/pull/30896#issuecomment-2043759408 @robertwb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [yaml] add use-case example [beam]

github-actions[bot] commented on PR #30896: URL: https://github.com/apache/beam/pull/30896#issuecomment-2043759925 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git

Re: [PR] [yaml] add use-case example [beam]

liferoad commented on code in PR #30896: URL: https://github.com/apache/beam/pull/30896#discussion_r1556506443 ## sdks/python/apache_beam/yaml/examples/simple_filter.yaml: ## @@ -0,0 +1,41 @@ +# coding=utf-8 Review Comment: are these yaml files packaged when releasing beam?

[PR] [yaml] remove Combine from yaml_experimental_features [beam]

Polber opened a new pull request, #30897: URL: https://github.com/apache/beam/pull/30897 Removes the need to set `--yaml_experimental_features=Combine` to run aggregations in Beam YAML Thank you for your contribution! Follow this checklist to help us

Re: [PR] [Python] Add a couple quality-of-life improvemenets to `testing.util.assert_that` [beam]

tvalentyn commented on PR #30771: URL: https://github.com/apache/beam/pull/30771#issuecomment-2043802447 pydantic errors should be resolved now. ptal at linter errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[PR] [yaml] Add Beam YAML Blog [beam]

Polber opened a new pull request, #30898: URL: https://github.com/apache/beam/pull/30898 Adds a Beam YAML announcement blog Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [

Re: [PR] Revert "Revert #30533: Automatically execute unbounded pipelines in streaming mode." [beam]

liferoad commented on code in PR #30894: URL: https://github.com/apache/beam/pull/30894#discussion_r1556620836 ## sdks/python/apache_beam/runners/dataflow/dataflow_runner.py: ## @@ -415,6 +416,12 @@ def run_pipeline(self, pipeline, options, pipeline_proto=None):

Re: [PR] [yaml] Add Beam YAML Blog [beam]

github-actions[bot] commented on PR #30898: URL: https://github.com/apache/beam/pull/30898#issuecomment-2043944172 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated

Re: [PR] [yaml] remove Combine from yaml_experimental_features [beam]

github-actions[bot] commented on PR #30897: URL: https://github.com/apache/beam/pull/30897#issuecomment-2043944210 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @jrmccluskey for label python. R: @liferoad for label website.

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

jbsabbagh commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1555950658 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java: ## @@ -191,6 +191,12 @@ private ReadFromKafkaDoFn(

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

liferoad commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1555976984 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java: ## @@ -587,6 +587,7 @@ public static Read read() {

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

jbsabbagh commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1555959733 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java: ## @@ -518,7 +525,7 @@ private ConsumerRecords poll( return

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

xianhualiu commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1556018716 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java: ## @@ -518,7 +525,7 @@ private ConsumerRecords poll( return

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

xianhualiu commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1556019131 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java: ## @@ -587,6 +587,7 @@ public static Read read() {

Re: [PR] Patch release website changes [beam]

damccorm merged PR #30839: URL: https://github.com/apache/beam/pull/30839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [Bug]: Cannot read from Kafka due to short poll timeout of consumer in KafkaIO [beam]

jbsabbagh commented on issue #30870: URL: https://github.com/apache/beam/issues/30870#issuecomment-2042957025 This also affects the Python SDK. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

jbsabbagh commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1555959733 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java: ## @@ -518,7 +525,7 @@ private ConsumerRecords poll( return

Re: [PR] [bug30870]: make consumer polling timeout configurable for KafkaIO.Read [beam]

xianhualiu commented on code in PR #30877: URL: https://github.com/apache/beam/pull/30877#discussion_r1556016519 ## sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java: ## @@ -191,6 +191,12 @@ private ReadFromKafkaDoFn(

Re: [I] [Feature Request]: Managed transforms Java API [beam]

ahmedabu98 closed issue #30830: [Feature Request]: Managed transforms Java API URL: https://github.com/apache/beam/issues/30830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] Clean doc related to write data in bigquery.py [beam]

kevinzous opened a new pull request, #30887: URL: https://github.com/apache/beam/pull/30887 * add missing closing parenthesis * add unique names to PTransform operations **Please** add a meaningful description for your change here Thank you

[PR] Add PubSubIO Stress test [beam]

akashorabek opened a new pull request, #30886: URL: https://github.com/apache/beam/pull/30886 This pull request introduces stress tests for PubSubIO, designed to assess the performance under various conditions. The stress tests simulate dynamic load increases and evaluate the behavior of

Re: [PR] [Java] ManagedIO [beam]

ahmedabu98 merged PR #30808: URL: https://github.com/apache/beam/pull/30808 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Duet AI Prompts - Documentation Lookup Without Links [beam]

andreydevyatkin commented on code in PR #30873: URL: https://github.com/apache/beam/pull/30873#discussion_r1555991358 ## learning/prompts/documentation-lookup-nolinks/09_basic_triggers.md: ## @@ -0,0 +1,30 @@ +Prompt: +What is a trigger in Apache Beam? + +Response: +Apache Beam