Re: [PR] Bump github.com/aws/aws-sdk-go-v2/credentials from 1.18.5 to 1.18.6 in /sdks [beam]

2025-08-21 Thread via GitHub
github-actions[bot] commented on PR #35939: URL: https://github.com/apache/beam/pull/35939#issuecomment-3213099586 Assigning reviewers: R: @lostluck for label go. Note: If you would like to opt out of this review, comment `assign to next reviewer`. Available commands

Re: [PR] Bump google.golang.org/protobuf from 1.36.7 to 1.36.8 in /sdks [beam]

2025-08-21 Thread via GitHub
github-actions[bot] commented on PR #35937: URL: https://github.com/apache/beam/pull/35937#issuecomment-3213099785 Assigning reviewers: R: @jrmccluskey for label go. Note: If you would like to opt out of this review, comment `assign to next reviewer`. Available comma

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.31.1 to 1.31.2 in /sdks [beam]

2025-08-21 Thread via GitHub
github-actions[bot] commented on PR #35938: URL: https://github.com/apache/beam/pull/35938#issuecomment-3213099698 Assigning reviewers: R: @shunping for label go. Note: If you would like to opt out of this review, comment `assign to next reviewer`. Available commands

[PR] Bump github.com/aws/aws-sdk-go-v2/credentials from 1.18.5 to 1.18.6 in /sdks [beam]

2025-08-21 Thread via GitHub
dependabot[bot] opened a new pull request, #35939: URL: https://github.com/apache/beam/pull/35939 Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.18.5 to 1.18.6. Changelog Sourced from https://github.com/aws/aws-sdk-go-v2/blob/config/v

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/credentials from 1.18.3 to 1.18.5 in /sdks [beam]

2025-08-21 Thread via GitHub
dependabot[bot] commented on PR #35925: URL: https://github.com/apache/beam/pull/35925#issuecomment-3213026382 Superseded by #35939. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/credentials from 1.18.3 to 1.18.5 in /sdks [beam]

2025-08-21 Thread via GitHub
dependabot[bot] closed pull request #35925: Bump github.com/aws/aws-sdk-go-v2/credentials from 1.18.3 to 1.18.5 in /sdks URL: https://github.com/apache/beam/pull/35925 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.29.18 to 1.31.1 in /sdks [beam]

2025-08-21 Thread via GitHub
dependabot[bot] commented on PR #35926: URL: https://github.com/apache/beam/pull/35926#issuecomment-3213022995 Superseded by #35938. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.29.18 to 1.31.1 in /sdks [beam]

2025-08-21 Thread via GitHub
dependabot[bot] closed pull request #35926: Bump github.com/aws/aws-sdk-go-v2/config from 1.29.18 to 1.31.1 in /sdks URL: https://github.com/apache/beam/pull/35926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.31.1 to 1.31.2 in /sdks [beam]

2025-08-21 Thread via GitHub
dependabot[bot] opened a new pull request, #35938: URL: https://github.com/apache/beam/pull/35938 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.31.1 to 1.31.2. Commits https://github.com/aws/aws-sdk-go-v2/commit/ba4ee4da236306b260326a

[PR] Bump google.golang.org/protobuf from 1.36.7 to 1.36.8 in /sdks [beam]

2025-08-21 Thread via GitHub
dependabot[bot] opened a new pull request, #35937: URL: https://github.com/apache/beam/pull/35937 Bumps google.golang.org/protobuf from 1.36.7 to 1.36.8. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=google.go

Re: [PR] [Go SDK] Add Write transform to cross-language Bigtable package - fixes #35873 [beam]

2025-08-21 Thread via GitHub
shunping commented on PR #35876: URL: https://github.com/apache/beam/pull/35876#issuecomment-3213009153 Sorry for the late reply. I am working to verify your fix. Will update early next week. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Allow users to pass service name for profiler for Java And Go SDK [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35903: URL: https://github.com/apache/beam/pull/35903#discussion_r2292516743 ## sdks/go/container/boot.go: ## @@ -61,22 +61,46 @@ const ( workerPoolIdEnv = "BEAM_GO_WORKER_POOL_ID" ) -func configureGoogleCloudProfilerE

Re: [PR] Allow users to pass service name for profiler for Java And Go SDK [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35903: URL: https://github.com/apache/beam/pull/35903#discussion_r2292529866 ## sdks/java/container/boot.go: ## @@ -196,25 +196,22 @@ func main() { enableGoogleCloudProfiler := strings.Contains(options, enableGoogleCloudProfilerOption)

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292594851 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/wordCountInclude.yaml: ## @@ -0,0 +1,66 @@ +# coding=utf-8 +# +# Licensed to the Apache Software Foun

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292581263 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/README.md: ## @@ -0,0 +1,63 @@ + + +## Jinja % include Pipeline + +This example leverages the `% incl

Re: [PR] Add a security GCP log analyzer [beam]

2025-08-21 Thread via GitHub
pabloem commented on code in PR #35922: URL: https://github.com/apache/beam/pull/35922#discussion_r2292594117 ## infra/security/config.yml: ## @@ -0,0 +1,43 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTIC

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292592673 ## sdks/python/apache_beam/yaml/examples/testing/input_data.py: ## @@ -33,6 +33,36 @@ ]) +def word_count_jinja_parameter_data(): + return \ +'{"readFromTex

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292593047 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/submodules/combineTransform.yaml: ## @@ -0,0 +1,27 @@ +# coding=utf-8 +# +# Licensed to the Apache So

Re: [PR] Add a security GCP log analyzer [beam]

2025-08-21 Thread via GitHub
pabloem commented on code in PR #35922: URL: https://github.com/apache/beam/pull/35922#discussion_r2292593422 ## infra/security/log_analyzer.py: ## @@ -0,0 +1,277 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the

Re: [PR] Add a security GCP log analyzer [beam]

2025-08-21 Thread via GitHub
pabloem commented on code in PR #35922: URL: https://github.com/apache/beam/pull/35922#discussion_r2292592687 ## infra/security/log_analyzer.py: ## @@ -0,0 +1,277 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292584730 ## sdks/python/apache_beam/yaml/examples/testing/input_data.py: ## @@ -33,6 +33,34 @@ def text_data(): ]) +def word_count_jinja_parameter_data(): + return \ +

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292592257 ## sdks/python/apache_beam/yaml/examples/testing/input_data.py: ## @@ -33,6 +33,36 @@ def text_data(): ]) +def word_count_jinja_parameter_data(): + return \ +

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292590837 ## sdks/python/apache_beam/yaml/examples/testing/examples_test.py: ## @@ -538,8 +558,64 @@ word = element.split('=')[1].split(',')[0].replace("'", '') count

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292582922 ## sdks/python/apache_beam/yaml/examples/testing/examples_test.py: ## @@ -1175,6 +1251,51 @@ def _batch_log_analysis_test_preprocessor( return test_spec +@YamlE

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292589161 ## sdks/python/apache_beam/yaml/examples/testing/examples_test.py: ## @@ -339,8 +343,21 @@ def test_yaml_example(self): for i, line in enumerate(expected):

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292588025 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/README.md: ## @@ -0,0 +1,63 @@ + + +## Jinja % include Pipeline + +This example leverages the `% incl

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
gemini-code-assist[bot] commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292587296 ## sdks/python/apache_beam/yaml/examples/testing/input_data.py: ## @@ -33,6 +33,36 @@ def text_data(): ]) +def word_count_jinja_parameter_data():

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on PR #35914: URL: https://github.com/apache/beam/pull/35914#issuecomment-3212872278 /gemini review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292582570 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/README.md: ## @@ -0,0 +1,63 @@ + + +## Jinja % include Pipeline + +This example leverages the `% incl

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292582347 ## sdks/python/apache_beam/yaml/examples/README.md: ## @@ -229,6 +233,27 @@ gcloud dataflow yaml run $JOB_NAME \ --region $REGION ``` +### Jinja + +Jinja [templ

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292581951 ## sdks/python/apache_beam/yaml/examples/testing/examples_test.py: ## @@ -538,8 +557,64 @@ word = element.split('=')[1].split(',')[0].replace("'", '') count

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292581637 ## sdks/python/apache_beam/yaml/examples/testing/input_data.py: ## @@ -33,6 +33,34 @@ ]) +def word_count_jinja_parameter_data(): + return \ +'{"readFromTex

Re: [PR] Add a security GCP log analyzer [beam]

2025-08-21 Thread via GitHub
pabloem commented on code in PR #35922: URL: https://github.com/apache/beam/pull/35922#discussion_r2292575421 ## infra/security/log_analyzer.py: ## @@ -0,0 +1,277 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.29.18 to 1.31.1 in /sdks [beam]

2025-08-21 Thread via GitHub
github-actions[bot] commented on PR #35926: URL: https://github.com/apache/beam/pull/35926#issuecomment-3212834165 Assigning reviewers: R: @jrmccluskey for label go. Note: If you would like to opt out of this review, comment `assign to next reviewer`. Available comma

Re: [PR] Bump actions/setup-java from 3 to 5 [beam-starter-java]

2025-08-21 Thread via GitHub
liferoad merged PR #150: URL: https://github.com/apache/beam-starter-java/pull/150 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@be

Re: [PR] Add a security GCP log analyzer [beam]

2025-08-21 Thread via GitHub
pabloem commented on code in PR #35922: URL: https://github.com/apache/beam/pull/35922#discussion_r2292556742 ## infra/security/log_analyzer.py: ## @@ -0,0 +1,277 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the

Re: [PR] GCP IAM Custom Roles [beam]

2025-08-21 Thread via GitHub
pabloem commented on PR #35107: URL: https://github.com/apache/beam/pull/35107#issuecomment-3212816448 okay - the contents of this PR do not perform the migration. They are the utilities necessary to perform it. These look good. My proposal for the migration is to do a small subset of pe

Re: [PR] GCP IAM Custom Roles [beam]

2025-08-21 Thread via GitHub
pabloem commented on PR #35107: URL: https://github.com/apache/beam/pull/35107#issuecomment-3212816814 I will merge this PR next Tuesday. FYI @damccorm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Allow users to pass service name for profiler for Java And Go SDK [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35903: URL: https://github.com/apache/beam/pull/35903#discussion_r2292529866 ## sdks/java/container/boot.go: ## @@ -196,25 +196,22 @@ func main() { enableGoogleCloudProfiler := strings.Contains(options, enableGoogleCloudProfilerOption)

Re: [PR] Allow users to pass service name for profiler for Java And Go SDK [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35903: URL: https://github.com/apache/beam/pull/35903#discussion_r2292529866 ## sdks/java/container/boot.go: ## @@ -196,25 +196,22 @@ func main() { enableGoogleCloudProfiler := strings.Contains(options, enableGoogleCloudProfilerOption)

Re: [PR] Allow users to pass service name for profiler for Java And Go SDK [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35903: URL: https://github.com/apache/beam/pull/35903#discussion_r2292522177 ## sdks/go/container/boot.go: ## @@ -119,6 +143,7 @@ func main() { log.Fatalf("Failed to obtain provisioning information: %v", err) } lo

Re: [PR] Allow users to pass service name for profiler for Java And Go SDK [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35903: URL: https://github.com/apache/beam/pull/35903#discussion_r2292526524 ## sdks/go/container/boot_test.go: ## @@ -205,57 +205,96 @@ func constructArtifactInformation(t *testing.T, roleUrn string, path string, sha } } +func clear

Re: [PR] Allow users to pass service name for profiler for Java And Go SDK [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35903: URL: https://github.com/apache/beam/pull/35903#discussion_r2292516743 ## sdks/go/container/boot.go: ## @@ -61,22 +61,46 @@ const ( workerPoolIdEnv = "BEAM_GO_WORKER_POOL_ID" ) -func configureGoogleCloudProfilerE

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.84 to 1.18.5 in /sdks [beam]

2025-08-21 Thread via GitHub
shunping merged PR #35928: URL: https://github.com/apache/beam/pull/35928 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] fix: support vector features in PyOD anomaly detection batching logic [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35884: URL: https://github.com/apache/beam/pull/35884#discussion_r2292488982 ## sdks/python/apache_beam/ml/anomaly/detectors/pyod_adapter.py: ## @@ -29,6 +35,7 @@ from apache_beam.ml.anomaly.specifiable import specifiable from apache_beam.ml.a

Re: [PR] fix: support vector features in PyOD anomaly detection batching logic [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35884: URL: https://github.com/apache/beam/pull/35884#discussion_r2292488982 ## sdks/python/apache_beam/ml/anomaly/detectors/pyod_adapter.py: ## @@ -29,6 +35,7 @@ from apache_beam.ml.anomaly.specifiable import specifiable from apache_beam.ml.a

Re: [PR] fix: support vector features in PyOD anomaly detection batching logic [beam]

2025-08-21 Thread via GitHub
shunping commented on code in PR #35884: URL: https://github.com/apache/beam/pull/35884#discussion_r2292488982 ## sdks/python/apache_beam/ml/anomaly/detectors/pyod_adapter.py: ## @@ -29,6 +35,7 @@ from apache_beam.ml.anomaly.specifiable import specifiable from apache_beam.ml.a

[PR] Bump actions/setup-java from 3 to 5 [beam-starter-java]

2025-08-21 Thread via GitHub
dependabot[bot] opened a new pull request, #150: URL: https://github.com/apache/beam-starter-java/pull/150 Bumps [actions/setup-java](https://github.com/actions/setup-java) from 3 to 5. Release notes Sourced from https://github.com/actions/setup-java/releases";>actions/setup-java's

[I] [Task]: YAML - update expand_jinja method [beam]

2025-08-21 Thread via GitHub
derrickaw opened a new issue, #35936: URL: https://github.com/apache/beam/issues/35936 ### What needs to happen? Update expand_jinja method to handle more jinja components. ### Issue Priority Priority: 2 (default / most normal work should be filed as P2) ### Issue

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on PR #35914: URL: https://github.com/apache/beam/pull/35914#issuecomment-3211890296 Run Python PreCommit 3.12 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] fix: support vector features in PyOD anomaly detection batching logic [beam]

2025-08-21 Thread via GitHub
mohamedawnallah commented on PR #35884: URL: https://github.com/apache/beam/pull/35884#issuecomment-3211189428 @vijay-x-Raj – The formatter CI pipeline passed now 🎉 The linter failing with this err (from publicly available CI logs) ``` apache_beam/ml/anomaly/detectors/pyod_adapte

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
codecov[bot] commented on PR #35914: URL: https://github.com/apache/beam/pull/35914#issuecomment-3211692830 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/35914?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292419646 ## sdks/python/apache_beam/yaml/examples/testing/examples_test.py: ## @@ -538,8 +557,64 @@ word = element.split('=')[1].split(',')[0].replace("'", '') count

Re: [PR] Fix Python Arm workflows [beam]

2025-08-21 Thread via GitHub
Abacn commented on PR #35900: URL: https://github.com/apache/beam/pull/35900#issuecomment-3211216713 requirements.txt are generated. The changes to them alone will gets overriden. Need to update setup.py then run `./gradlew :sdks:python:container:generatePythonRequirementsAll` -- This is

Re: [PR] increase grpc timeouts & force seq exec (#34739) [beam]

2025-08-21 Thread via GitHub
damccorm merged PR #35915: URL: https://github.com/apache/beam/pull/35915 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] [Postrelease-2.67] Remove mysql-connector-python dependency [beam]

2025-08-21 Thread via GitHub
github-actions[bot] commented on PR #35933: URL: https://github.com/apache/beam/pull/35933#issuecomment-3211277247 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewer

[I] [Task]: Update Python slowly updating side input example [beam]

2025-08-21 Thread via GitHub
damccorm opened a new issue, #35935: URL: https://github.com/apache/beam/issues/35935 ### What needs to happen? https://beam.apache.org/documentation/patterns/side-inputs/ is missing a side input pattern for Python When doing this, we should factor in https://github.com/apache

Re: [PR] Add GitHub Actions workflow for managing GCP Service Account keys [beam]

2025-08-21 Thread via GitHub
pabloem commented on code in PR #35911: URL: https://github.com/apache/beam/pull/35911#discussion_r2292309472 ## .github/workflows/beam_Infrastructure_ServiceAccountKeys.yml: ## @@ -0,0 +1,68 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] Add GitHub Actions workflow for managing GCP Service Account keys [beam]

2025-08-21 Thread via GitHub
pabloem commented on code in PR #35911: URL: https://github.com/apache/beam/pull/35911#discussion_r2292305607 ## .github/workflows/beam_Infrastructure_ServiceAccountKeys.yml: ## @@ -0,0 +1,68 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] Infra enforcer gha [beam]

2025-08-21 Thread via GitHub
pabloem commented on code in PR #35910: URL: https://github.com/apache/beam/pull/35910#discussion_r2292301412 ## .github/workflows/beam_Infrastructure_PolicyEnforcer.yml: ## @@ -0,0 +1,83 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lic

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292063929 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/wordCountInclude.yaml: ## @@ -0,0 +1,71 @@ +# coding=utf-8 +# +# Licensed to the Apache Software Foun

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
liferoad commented on PR #35914: URL: https://github.com/apache/beam/pull/35914#issuecomment-3212176767 I am trying to test how good Gemini knows our YAML. The review comments could be skipped. :) -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
gemini-code-assist[bot] commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292205965 ## sdks/python/apache_beam/yaml/examples/testing/input_data.py: ## @@ -33,6 +33,34 @@ def text_data(): ]) +def word_count_jinja_parameter_data():

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
liferoad commented on PR #35914: URL: https://github.com/apache/beam/pull/35914#issuecomment-3212175152 /gemini review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Adds a pipeline option to terminate processing of an element after a timeout (python) [beam]

2025-08-21 Thread via GitHub
scwhittle commented on code in PR #35391: URL: https://github.com/apache/beam/pull/35391#discussion_r2291731430 ## sdks/python/apache_beam/runners/worker/sdk_worker_main.py: ## @@ -219,6 +224,14 @@ def main(unused_argv): fn_log_handler.close() +def flush_fn_log_handle

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
damccorm commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292082106 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/wordCountInclude.yaml: ## @@ -0,0 +1,71 @@ +# coding=utf-8 +# +# Licensed to the Apache Software Found

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292099853 ## sdks/python/apache_beam/yaml/examples/README.md: ## @@ -229,6 +233,27 @@ gcloud dataflow yaml run $JOB_NAME \ --region $REGION ``` +### Jinja + +Jinja [templ

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292094401 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/wordCountInclude.yaml: ## @@ -0,0 +1,71 @@ +# coding=utf-8 +# +# Licensed to the Apache Software Foun

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
damccorm commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292087497 ## sdks/python/apache_beam/yaml/examples/README.md: ## @@ -229,6 +233,27 @@ gcloud dataflow yaml run $JOB_NAME \ --region $REGION ``` +### Jinja + +Jinja [templa

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
damccorm commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292084152 ## sdks/python/apache_beam/yaml/examples/README.md: ## @@ -229,6 +233,27 @@ gcloud dataflow yaml run $JOB_NAME \ --region $REGION ``` +### Jinja + +Jinja [templa

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292063929 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/wordCountInclude.yaml: ## @@ -0,0 +1,71 @@ +# coding=utf-8 +# +# Licensed to the Apache Software Foun

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292059002 ## sdks/python/apache_beam/yaml/examples/README.md: ## @@ -229,6 +233,27 @@ gcloud dataflow yaml run $JOB_NAME \ --region $REGION ``` +### Jinja + +Jinja [templ

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292056256 ## sdks/python/apache_beam/yaml/examples/README.md: ## @@ -229,6 +233,27 @@ gcloud dataflow yaml run $JOB_NAME \ --region $REGION ``` +### Jinja + +Jinja [templ

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292063929 ## sdks/python/apache_beam/yaml/examples/transforms/jinja/include/wordCountInclude.yaml: ## @@ -0,0 +1,71 @@ +# coding=utf-8 +# +# Licensed to the Apache Software Foun

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292056256 ## sdks/python/apache_beam/yaml/examples/README.md: ## @@ -229,6 +233,27 @@ gcloud dataflow yaml run $JOB_NAME \ --region $REGION ``` +### Jinja + +Jinja [templ

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
damccorm commented on code in PR #35914: URL: https://github.com/apache/beam/pull/35914#discussion_r2292036590 ## sdks/python/apache_beam/yaml/examples/README.md: ## @@ -229,6 +233,27 @@ gcloud dataflow yaml run $JOB_NAME \ --region $REGION ``` +### Jinja + +Jinja [templa

Re: [PR] python: replace <=3.12 with <3.13 for google-apitools marker (semanti… [beam]

2025-08-21 Thread via GitHub
diegofhdz commented on PR #35929: URL: https://github.com/apache/beam/pull/35929#issuecomment-3211841986 assign set of reviewers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Add AftersynchronizedProcessing Time as continuation trigger [beam]

2025-08-21 Thread via GitHub
derrickaw commented on PR #35913: URL: https://github.com/apache/beam/pull/35913#issuecomment-3211935296 Run Python_Transforms PreCommit 3.12 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] [bugFix] sdks/python: properly add Milvus as extra dependency [beam]

2025-08-21 Thread via GitHub
damccorm commented on PR #35920: URL: https://github.com/apache/beam/pull/35920#issuecomment-3211926155 Hm, I see the following error logs in the postcommit python: ``` "">cls = @classmethod def setUpClass(cls): cls._db = MilvusEnrichmentTestHelper.start_db_container(cls._ve

Re: [PR] [DO NOT MERGE] Prototype Vertex MultiModal embedding handler [beam]

2025-08-21 Thread via GitHub
damccorm commented on code in PR #35677: URL: https://github.com/apache/beam/pull/35677#discussion_r2292012846 ## sdks/python/apache_beam/ml/transforms/embeddings/vertex_ai.py: ## @@ -281,3 +294,194 @@ def get_ptransform_for_processing(self, **kwargs) -> beam.PTransform: r

Re: [PR] increase grpc timeouts & force seq exec (#34739) [beam]

2025-08-21 Thread via GitHub
damccorm commented on code in PR #35915: URL: https://github.com/apache/beam/pull/35915#discussion_r2291341265 ## .github/workflows/beam_PreCommit_Python.yml: ## @@ -53,6 +53,23 @@ env: DEVELOCITY_ACCESS_KEY: ${{ secrets.DEVELOCITY_ACCESS_KEY }} GRADLE_ENTERPRISE_CACHE_USE

Re: [PR] python: replace <=3.12 with <3.13 for google-apitools marker (semanti… [beam]

2025-08-21 Thread via GitHub
github-actions[bot] commented on PR #35929: URL: https://github.com/apache/beam/pull/35929#issuecomment-3211844697 Assigning reviewers: R: @claudevdm for label python. Note: If you would like to opt out of this review, comment `assign to next reviewer`. Available com

Re: [PR] Remove mysql-connector-python dependency [beam]

2025-08-21 Thread via GitHub
codecov[bot] commented on PR #35932: URL: https://github.com/apache/beam/pull/35932#issuecomment-3211519120 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/35932?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term

[I] [Bug]: Latest.Globally() issues in global window [beam]

2025-08-21 Thread via GitHub
damccorm opened a new issue, #35934: URL: https://github.com/apache/beam/issues/35934 ### What happened? I was trying to repro an issue with slowly updating side inputs, and I found that the following code only logs one element: ``` pipeline = beam.Pipeline(options=options)

Re: [PR] [YAML] - Jinja % include example [beam]

2025-08-21 Thread via GitHub
derrickaw commented on PR #35914: URL: https://github.com/apache/beam/pull/35914#issuecomment-3211640103 Run Python_Transforms PreCommit 3.10 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] [Postrelease-2.67] Remove mysql-connector-python dependency [beam]

2025-08-21 Thread via GitHub
Abacn commented on PR #35933: URL: https://github.com/apache/beam/pull/35933#issuecomment-3211552386 ``` apache_beam/yaml/integration_tests.py::RuninferenceTest::test_only - google.api_core.exceptions.PermissionDenied: 403 Tuned endpoint is blocked because publisher model `projects/8441

Re: [PR] Add AftersynchronizedProcessing Time as continuation trigger [beam]

2025-08-21 Thread via GitHub
tarun-google commented on code in PR #35913: URL: https://github.com/apache/beam/pull/35913#discussion_r2291714497 ## sdks/python/apache_beam/transforms/trigger.py: ## @@ -317,6 +317,17 @@ def from_runner_api(proto, context): def to_runner_api(self, unused_context): pass

Re: [PR] [Python] Deprecate --enable_streaming_engine flag since it is always enabled. [beam]

2025-08-21 Thread via GitHub
scwhittle merged PR #35917: URL: https://github.com/apache/beam/pull/35917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apach

Re: [PR] [Python] Deprecate --enable_streaming_engine flag since it is always enabled. [beam]

2025-08-21 Thread via GitHub
scwhittle commented on PR #35917: URL: https://github.com/apache/beam/pull/35917#issuecomment-3211473876 No longer formatting errors, the test failure appears unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Add AftersynchronizedProcessing Time as continuation trigger [beam]

2025-08-21 Thread via GitHub
derrickaw commented on code in PR #35913: URL: https://github.com/apache/beam/pull/35913#discussion_r2291693185 ## sdks/python/apache_beam/transforms/trigger.py: ## @@ -317,6 +317,17 @@ def from_runner_api(proto, context): def to_runner_api(self, unused_context): pass

[PR] Remove mysql-connector-python dependency [beam]

2025-08-21 Thread via GitHub
Abacn opened a new pull request, #35932: URL: https://github.com/apache/beam/pull/35932 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly

Re: [PR] Add AftersynchronizedProcessing Time as continuation trigger [beam]

2025-08-21 Thread via GitHub
tarun-google commented on code in PR #35913: URL: https://github.com/apache/beam/pull/35913#discussion_r2291670785 ## sdks/python/apache_beam/transforms/trigger.py: ## @@ -317,6 +317,17 @@ def from_runner_api(proto, context): def to_runner_api(self, unused_context): pass

Re: [PR] add a jinja % include pipeline example to docs [beam]

2025-08-21 Thread via GitHub
github-actions[bot] commented on PR #35931: URL: https://github.com/apache/beam/pull/35931#issuecomment-3211346837 Assigning reviewers: R: @liferoad for label website. Note: If you would like to opt out of this review, comment `assign to next reviewer`. Available com

Re: [PR] fix: support vector features in PyOD anomaly detection batching logic [beam]

2025-08-21 Thread via GitHub
vijay-x-Raj commented on PR #35884: URL: https://github.com/apache/beam/pull/35884#issuecomment-3211323974 yea I will try that and after will keep trying !. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Remove mysql-connector-python dependency [beam]

2025-08-21 Thread via GitHub
github-actions[bot] commented on PR #35932: URL: https://github.com/apache/beam/pull/35932#issuecomment-3211232444 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewer

[PR] Remove mysql-connector-python dependency [beam]

2025-08-21 Thread via GitHub
Abacn opened a new pull request, #35933: URL: https://github.com/apache/beam/pull/35933 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly

Re: [PR] [Postrelease-2.67] Remove mysql-connector-python dependency [beam]

2025-08-21 Thread via GitHub
Abacn commented on PR #35933: URL: https://github.com/apache/beam/pull/35933#issuecomment-3211253412 R: @damccorm @derrickaw cc: @kennknowles @Amar3tto -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Remove mysql-connector-python dependency [beam]

2025-08-21 Thread via GitHub
Abacn commented on PR #35932: URL: https://github.com/apache/beam/pull/35932#issuecomment-3211229099 R: @damccorm @derrickaw cc: @kennknowles @Amar3tto -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Runner support status for Beam 3 [beam]

2025-08-21 Thread via GitHub
Abacn merged PR #35908: URL: https://github.com/apache/beam/pull/35908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.or

Re: [PR] increase grpc timeouts & force seq exec (#34739) [beam]

2025-08-21 Thread via GitHub
A1K28 commented on code in PR #35915: URL: https://github.com/apache/beam/pull/35915#discussion_r2291291191 ## .github/workflows/beam_PreCommit_Python.yml: ## @@ -53,6 +53,23 @@ env: DEVELOCITY_ACCESS_KEY: ${{ secrets.DEVELOCITY_ACCESS_KEY }} GRADLE_ENTERPRISE_CACHE_USERNA

  1   2   >