Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb merged PR #30976: URL: https://github.com/apache/beam/pull/30976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1593206728 ## sdks/python/apache_beam/yaml/main.py: ## @@ -64,9 +72,19 @@ def _pipeline_spec_from_args(known_args): return pipeline_yaml +class

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1588009137 ## sdks/python/apache_beam/yaml/main.py: ## @@ -64,9 +72,19 @@ def _pipeline_spec_from_args(known_args): return pipeline_yaml +class

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1588005108 ## sdks/python/apache_beam/yaml/main.py: ## @@ -64,9 +72,19 @@ def _pipeline_spec_from_args(known_args): return pipeline_yaml +class

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

Polber commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2061800255 > Given the uncertainty with what's going on here, another option is to put the changes in to the dataflow template itself in the short term. I'll draft up a PR for that -- This

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2061769275 Given the uncertainty with what's going on here, another option is to put the changes in to the dataflow template itself in the short term. -- This is an automated message from the

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1569161181 ## sdks/python/apache_beam/yaml/main.py: ## @@ -64,9 +72,19 @@ def _pipeline_spec_from_args(known_args): return pipeline_yaml +class

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2061683074 So I'm still trying to figure out why these same tests pass in isolation ( https://github.com/apache/beam/actions/runs/8722508185/job/23928542728?pr=30976 ) but fail when all tests are

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2061635197 Oh whoops, missed https://github.com/apache/beam/pull/30976/commits/91d4dc1af683c87808048bc03ed180522b12c13d - thanks for catching -- This is an automated message from the Apache Git

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2061227644 (resolved some conflicts and cleaned up CHANGES.md while at it) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2060017241 A merge to master seems to have resolved the row coder issues. Running a full suite of tests again. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2059831558 Precommit failures look real? Its not failing on master, but I also don't see any relevant changes here -

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1567737236 ## sdks/python/apache_beam/yaml/main.py: ## @@ -45,9 +47,45 @@ def _configure_parser(argv): help='none: do no pipeline validation against the schema; '

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

Polber commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1567679576 ## sdks/python/apache_beam/yaml/main.py: ## @@ -45,9 +47,45 @@ def _configure_parser(argv): help='none: do no pipeline validation against the schema; '

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

Polber commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1567678666 ## sdks/python/apache_beam/yaml/main.py: ## @@ -45,9 +47,45 @@ def _configure_parser(argv): help='none: do no pipeline validation against the schema; '

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

Polber commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1567678666 ## sdks/python/apache_beam/yaml/main.py: ## @@ -45,9 +47,45 @@ def _configure_parser(argv): help='none: do no pipeline validation against the schema; '

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1567611320 ## sdks/python/apache_beam/yaml/main.py: ## @@ -45,9 +47,45 @@ def _configure_parser(argv): help='none: do no pipeline validation against the schema; '

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1567607357 ## sdks/python/apache_beam/yaml/main_test.py: ## @@ -0,0 +1,91 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

Polber commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1567599364 ## sdks/python/apache_beam/yaml/main.py: ## @@ -45,9 +47,45 @@ def _configure_parser(argv): help='none: do no pipeline validation against the schema; '

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1567500942 ## sdks/python/apache_beam/yaml/main_test.py: ## @@ -0,0 +1,91 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2058043075 RowCoderTest, YamlIntegration, and Docs issues seem pre-existing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2057977117 isort wants to put jinja2 after apache_beam, e.g. ``` import apache_beam as beam +import jinja2 ``` Not sure how to resolve that. -- This is an automated message

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566446325 ## sdks/python/apache_beam/yaml/main.py: ## @@ -67,6 +105,12 @@ def _pipeline_spec_from_args(known_args): def run(argv=None): known_args, pipeline_args =

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566419436 ## sdks/python/apache_beam/yaml/main.py: ## @@ -67,6 +105,12 @@ def _pipeline_spec_from_args(known_args): def run(argv=None): known_args, pipeline_args =

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566374434 ## sdks/python/apache_beam/yaml/main.py: ## @@ -67,6 +105,12 @@ def _pipeline_spec_from_args(known_args): def run(argv=None): known_args, pipeline_args =

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566365704 ## sdks/python/apache_beam/yaml/main.py: ## @@ -45,9 +47,45 @@ def _configure_parser(argv): help='none: do no pipeline validation against the schema; '

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566346899 ## sdks/python/apache_beam/yaml/main.py: ## @@ -45,9 +47,45 @@ def _configure_parser(argv): help='none: do no pipeline validation against the schema; '

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566345277 ## sdks/python/apache_beam/yaml/main.py: ## @@ -67,6 +105,12 @@ def _pipeline_spec_from_args(known_args): def run(argv=None): known_args, pipeline_args =

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566345277 ## sdks/python/apache_beam/yaml/main.py: ## @@ -67,6 +105,12 @@ def _pipeline_spec_from_args(known_args): def run(argv=None): known_args, pipeline_args =

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566344279 ## sdks/python/apache_beam/yaml/main.py: ## @@ -67,6 +105,12 @@ def _pipeline_spec_from_args(known_args): def run(argv=None): known_args, pipeline_args =

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566344084 ## sdks/python/apache_beam/yaml/main.py: ## @@ -67,6 +105,12 @@ def _pipeline_spec_from_args(known_args): def run(argv=None): known_args, pipeline_args =

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566343526 ## sdks/python/apache_beam/yaml/main.py: ## @@ -67,6 +105,12 @@ def _pipeline_spec_from_args(known_args): def run(argv=None): known_args, pipeline_args =

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

damccorm commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566338553 ## sdks/python/apache_beam/yaml/main.py: ## @@ -45,9 +47,45 @@ def _configure_parser(argv): help='none: do no pipeline validation against the schema; '

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

Polber commented on code in PR #30976: URL: https://github.com/apache/beam/pull/30976#discussion_r1566336507 ## sdks/python/apache_beam/yaml/main.py: ## @@ -67,6 +105,12 @@ def _pipeline_spec_from_args(known_args): def run(argv=None): known_args, pipeline_args =

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

github-actions[bot] commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2057623102 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git

[PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb opened a new pull request, #30976: URL: https://github.com/apache/beam/pull/30976 This will allow Beam YAML to be used for various template applications. Thank you for your contribution! Follow this checklist to help us incorporate your

Re: [PR] [YAML] Add the ability to pre-process yaml files with jinja2. [beam]

robertwb commented on PR #30976: URL: https://github.com/apache/beam/pull/30976#issuecomment-2057621210 R: @Polber @damccorm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific