(beam) branch asf-site updated: Publishing website 2024/03/13 05:37:57 at commit 289bbc4
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new e3cb87940db Publishing website 2024/03/13 05:37:57 at commit 289bbc4 e3cb87940db is described below commit e3cb87940db9b7425bd29a6cc85f63d35fefaf06 Author: runner AuthorDate: Wed Mar 13 05:37:57 2024 + Publishing website 2024/03/13 05:37:57 at commit 289bbc4 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 1a0e1cd181e..d811aaa1852 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.54.0/2024-03-12T19:04:55-04:00/categories/blog/2024-03-12T19:04:55-04:00/blog/2024-03-12T19:04:55-04:00/categories/2024-03-12T19:04:55-04:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.54.0/2024-03-12T21:58:19-04:00/categories/blog/2024-03-12T21:58:19-04:00/blog/2024-03-12T21:58:19-04:00/categories/2024-03-12T21:58:19-04:00/catego [...] \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new cda1666065c Updating config from bot cda1666065c is described below commit cda1666065c24704b38a7baf915693c17ce2ecdc Author: github-actions AuthorDate: Wed Mar 13 05:34:19 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index af7cb15fe44..43aae0b9018 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -5,6 +5,6 @@ "jrmccluskey": 1709667439988, "youngoli": 1657688896155, "damccorm": 1680501930289, -"riteshghorse": 1709283976060 +"riteshghorse": 1710308057456 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 42eb97d3ecb Updating config from bot
42eb97d3ecb is described below
commit 42eb97d3ecb3b3f583f462ed172f11b0a8862922
Author: github-actions
AuthorDate: Wed Mar 13 05:34:18 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-30620.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30620.json
b/scripts/ci/pr-bot/state/pr-state/pr-30620.json
new file mode 100644
index 000..aa3638a74ec
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30620.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "riteshghorse"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch dependabot/go_modules/sdks/github.com/nats-io/nats-server/v2-2.10.11 deleted (was a3e85a4e39e)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/nats-io/nats-server/v2-2.10.11 in repository https://gitbox.apache.org/repos/asf/beam.git was a3e85a4e39e Bump github.com/nats-io/nats-server/v2 from 2.10.10 to 2.10.11 in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch dependabot/go_modules/sdks/github.com/nats-io/nats-server/v2-2.10.12 created (now 47252e90f55)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/nats-io/nats-server/v2-2.10.12 in repository https://gitbox.apache.org/repos/asf/beam.git at 47252e90f55 Bump github.com/nats-io/nats-server/v2 from 2.10.10 to 2.10.12 in /sdks No new revisions were added by this update.
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new d3a59db25e2 Updating config from bot
d3a59db25e2 is described below
commit d3a59db25e240c60f49de4ff52833e4a0968c970
Author: github-actions
AuthorDate: Wed Mar 13 04:34:28 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-30619.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30619.json
b/scripts/ci/pr-bot/state/pr-state/pr-30619.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30619.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch nightly-refs/heads/master updated (d22a7e783ab -> 289bbc45c56)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from d22a7e783ab Make defaults for optional SchemaTransformProvider methods
(#30560)
add 35252b92c0f Duet AI documentation lookup prompts for Beam ML (#30351)
add 0c8b78a9ff0 Remove Anand, add Shunping to reviewers (#30608)
add 97f8dae92f2 Put an explanatory comment in trigger files that exist
add 9a6777ebe7e Merge pull request #30586: Put an explanatory comment in
trigger files that exist
add 986f67fd92b Update blog template with known issues link to CHANGES
(#30611)
add 5a9a7d2f85b Add a known issue for gcsio in the notes of 2.53.0 and
2.54.0. (#30615)
add 06c76b4e930 Updates the test suite to use the transform service
(#30605)
add 17bbe8262b9 Remove MapState and SetState tests that mix v1 and v2
codepaths erroneously
add 8945b8fc144 Remove GroupIntoBatches tests that mix v1 and v2 codepaths
erroneously
add bf5551bd118 Remove PubsubSink tests that conflate v1 and v2 codepaths
erroneously
add 0a0186de555 Skip v1 translation steps for pipelines that will run on v2
add c79b6e157dd Merge pull request #30604: Skip running Dataflow v1 logic
when a job will run on v2
add 289bbc45c56 [Python] Pin keras version for tensorflow inference test
(#30614)
No new revisions were added by this update.
Summary of changes:
.github/REVIEWERS.yml | 5 +-
.../beam_PostCommit_Java_Avro_Versions.json| 3 +
.../beam_PostCommit_Java_DataflowV1.json | 3 +
.../beam_PostCommit_Java_PVR_Spark3_Streaming.json | 3 +
.../beam_PostCommit_Java_PVR_Spark_Batch.json | 3 +
...ostCommit_Java_ValidatesRunner_Dataflow_V2.json | 3 +
...Java_ValidatesRunner_Dataflow_V2_Streaming.json | 3 +
...stCommit_Java_ValidatesRunner_Flink_Java11.json | 3 +
...a_ValidatesRunner_SparkStructuredStreaming.json | 3 +
...m_PostCommit_Java_ValidatesRunner_Twister2.json | 3 +
.../beam_PostCommit_Java_ValidatesRunner_ULR.json | 3 +
...o_Versions.json => beam_PostCommit_Python.json} | 0
.../beam_PostCommit_Python_Examples_Direct.json| 3 +
.../beam_PostCommit_Python_Xlang_Gcp_Dataflow.json | 3 +
.../beam_PostCommit_Python_Xlang_Gcp_Direct.json | 4 +-
.../beam_PostCommit_Python_Xlang_IO_Dataflow.json | 3 +
.../beam_PostCommit_TransformService_Direct.json | 3 +
.../trigger_files/beam_PostCommit_XVR_Flink.json | 3 +
...PostCommit_XVR_PythonUsingJavaSQL_Dataflow.json | 3 +
CHANGES.md | 3 +
contributor-docs/release-guide.md | 2 +
.../33_ai_ml_pipelines.md | 22 +
.../36_ai_kfp_orchestration.md | 51 ++
.../37_ai_tfx_orchestration.md | 20
.../38_ai_runinference.md | 37
.../39_ai_runinference_metrics.md | 38
.../40_ai_bigquery_ml.md | 48 ++
.../41_ai_model_refresh.md | 42 +
.../42_ai_custom_inference.md | 50 ++
.../43_ai_llm_inference.md | 60
.../documentation-lookup/33_ai_ml_pipelines.md | 15 +++
.../36_ai_kfp_orchestration.md | 51 ++
.../37_ai_tfx_orchestration.md | 20
.../documentation-lookup/38_ai_runinference.md | 39
.../39_ai_runinference_metrics.md | 39
.../documentation-lookup/40_ai_bigquery_ml.md | 52 ++
.../documentation-lookup/41_ai_model_refresh.md| 42 +
.../documentation-lookup/42_ai_custom_inference.md | 50 ++
.../documentation-lookup/43_ai_llm_inference.md| 60
.../dataflow/DataflowPipelineTranslator.java | 6 +-
.../beam/runners/dataflow/DataflowRunner.java | 11 ++-
.../dataflow/DataflowPipelineTranslatorTest.java | 105 -
.../beam/runners/dataflow/DataflowRunnerTest.java | 29 +-
.../apache_beam/io/gcp/bigtableio_it_test.py | 16 +++-
.../ml/inference/tensorflow_tests_requirements.txt | 1 +
sdks/python/expansion-service-container/boot.go| 24 -
46 files changed, 845 insertions(+), 145 deletions(-)
copy .github/trigger_files/{beam_PostCommit_Java_Avro_Versions.json =>
beam_PostCommit_Python.json} (100%)
create mode 100644
.github/trigger_files/beam_PostCommit_TransformService_Direct.json
create mode 100644 .github/trigger_files/beam_PostCommit_XVR_Flink.json
create mode 100644
learning/prompts/documentation-lookup-nolinks/33_ai_ml_pipelines.md
create mode 100644
learning/prompts/documentation-lookup-nolinks/36_ai_kfp_orchestration.md
create mode 100644
learning/prompts/documentation-lookup-noli
(beam) branch master updated: [Python] Pin keras version for tensorflow inference test (#30614)
This is an automated email from the ASF dual-hosted git repository. riteshghorse pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 289bbc45c56 [Python] Pin keras version for tensorflow inference test (#30614) 289bbc45c56 is described below commit 289bbc45c56a0738e6e0f3c9ba17873d0d1a234c Author: Ritesh Ghorse AuthorDate: Tue Mar 12 21:58:19 2024 -0400 [Python] Pin keras version for tensorflow inference test (#30614) --- .github/trigger_files/beam_PostCommit_Python.json | 0 sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt | 1 + 2 files changed, 1 insertion(+) diff --git a/.github/trigger_files/beam_PostCommit_Python.json b/.github/trigger_files/beam_PostCommit_Python.json new file mode 100644 index 000..e69de29bb2d diff --git a/sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt b/sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt index 8195059a6c1..aebddf61226 100644 --- a/sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt +++ b/sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt @@ -15,6 +15,7 @@ # limitations under the License. # +keras<3.0.0 tensorflow>=2.12.0 tensorflow_hub>=0.10.0 Pillow>=9.0.0
(beam) branch master updated (06c76b4e930 -> c79b6e157dd)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 06c76b4e930 Updates the test suite to use the transform service (#30605) add 17bbe8262b9 Remove MapState and SetState tests that mix v1 and v2 codepaths erroneously add 8945b8fc144 Remove GroupIntoBatches tests that mix v1 and v2 codepaths erroneously add bf5551bd118 Remove PubsubSink tests that conflate v1 and v2 codepaths erroneously add 0a0186de555 Skip v1 translation steps for pipelines that will run on v2 new c79b6e157dd Merge pull request #30604: Skip running Dataflow v1 logic when a job will run on v2 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../dataflow/DataflowPipelineTranslator.java | 6 +- .../beam/runners/dataflow/DataflowRunner.java | 11 ++- .../dataflow/DataflowPipelineTranslatorTest.java | 105 - .../beam/runners/dataflow/DataflowRunnerTest.java | 29 +- 4 files changed, 16 insertions(+), 135 deletions(-)
(beam) 01/01: Merge pull request #30604: Skip running Dataflow v1 logic when a job will run on v2
This is an automated email from the ASF dual-hosted git repository. kenn pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit c79b6e157dd2f31ff6131938bade3e60a51a826c Merge: 06c76b4e930 0a0186de555 Author: Kenneth Knowles AuthorDate: Tue Mar 12 21:04:40 2024 -0400 Merge pull request #30604: Skip running Dataflow v1 logic when a job will run on v2 .../dataflow/DataflowPipelineTranslator.java | 6 +- .../beam/runners/dataflow/DataflowRunner.java | 11 ++- .../dataflow/DataflowPipelineTranslatorTest.java | 105 - .../beam/runners/dataflow/DataflowRunnerTest.java | 29 +- 4 files changed, 16 insertions(+), 135 deletions(-)
(beam) branch master updated: Updates the test suite to use the transform service (#30605)
This is an automated email from the ASF dual-hosted git repository.
chamikara pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 06c76b4e930 Updates the test suite to use the transform service
(#30605)
06c76b4e930 is described below
commit 06c76b4e930d76c6e38a9a78fce3ccd223de4cbd
Author: Chamikara Jayalath
AuthorDate: Tue Mar 12 17:00:30 2024 -0700
Updates the test suite to use the transform service (#30605)
* Updates the test suite to use the transform service
* Trigger the test suite
* Fix formatting
* Addressing reviewer comments
---
.../beam_PostCommit_TransformService_Direct.json | 3 +++
.../apache_beam/io/gcp/bigtableio_it_test.py | 16 +--
sdks/python/expansion-service-container/boot.go| 24 +-
3 files changed, 36 insertions(+), 7 deletions(-)
diff --git a/.github/trigger_files/beam_PostCommit_TransformService_Direct.json
b/.github/trigger_files/beam_PostCommit_TransformService_Direct.json
new file mode 100644
index 000..c4edaa85a89
--- /dev/null
+++ b/.github/trigger_files/beam_PostCommit_TransformService_Direct.json
@@ -0,0 +1,3 @@
+{
+ "comment": "Modify this file in a trivial way to cause this test suite to
run"
+}
diff --git a/sdks/python/apache_beam/io/gcp/bigtableio_it_test.py
b/sdks/python/apache_beam/io/gcp/bigtableio_it_test.py
index 8c0ca29c410..2445fdd5fcc 100644
--- a/sdks/python/apache_beam/io/gcp/bigtableio_it_test.py
+++ b/sdks/python/apache_beam/io/gcp/bigtableio_it_test.py
@@ -93,6 +93,11 @@ class TestReadFromBigTableIT(unittest.TestCase):
self.table = self.instance.table(self.TABLE_ID)
self.table.create()
_LOGGER.info("Created table [%s]", self.table.table_id)
+if (os.environ.get('TRANSFORM_SERVICE_PORT')):
+ self._transform_service_address = (
+ 'localhost:' + os.environ.get('TRANSFORM_SERVICE_PORT'))
+else:
+ self._transform_service_address = None
def tearDown(self):
try:
@@ -142,7 +147,8 @@ class TestReadFromBigTableIT(unittest.TestCase):
| bigtableio.ReadFromBigtable(
project_id=self.project,
instance_id=self.instance.instance_id,
- table_id=self.table.table_id)
+ table_id=self.table.table_id,
+ expansion_service=self._transform_service_address)
| "Extract cells" >> beam.Map(lambda row: row._cells))
assert_that(cells, equal_to(expected_cells))
@@ -190,6 +196,11 @@ class TestWriteToBigtableXlangIT(unittest.TestCase):
(self.TABLE_ID, str(int(time.time())), secrets.token_hex(3)))
self.table.create()
_LOGGER.info("Created table [%s]", self.table.table_id)
+if (os.environ.get('TRANSFORM_SERVICE_PORT')):
+ self._transform_service_address = (
+ 'localhost:' + os.environ.get('TRANSFORM_SERVICE_PORT'))
+else:
+ self._transform_service_address = None
def tearDown(self):
try:
@@ -216,7 +227,8 @@ class TestWriteToBigtableXlangIT(unittest.TestCase):
project_id=self.project,
instance_id=self.instance.instance_id,
table_id=self.table.table_id,
- use_cross_language=True))
+ use_cross_language=True,
+ expansion_service=self._transform_service_address))
def test_set_mutation(self):
row1: DirectRow = DirectRow('key-1')
diff --git a/sdks/python/expansion-service-container/boot.go
b/sdks/python/expansion-service-container/boot.go
index ba56b349c4e..a04b176f5d8 100644
--- a/sdks/python/expansion-service-container/boot.go
+++ b/sdks/python/expansion-service-container/boot.go
@@ -97,8 +97,8 @@ func installExtraPackages(requirementsFile string) error {
return nil
}
-func getUpdatedRequirementsFile(oldRequirementsFileName string,
dependenciesDir string) (string, error) {
- oldExtraPackages, err := getLines(filepath.Join(dependenciesDir,
oldRequirementsFileName))
+func getUpdatedRequirementsFile(oldDependenciesRequirementsFile string,
dependenciesDir string) (string, error) {
+ oldExtraPackages, err := getLines(oldDependenciesRequirementsFile)
if err != nil {
return "", err
}
@@ -145,9 +145,20 @@ func launchExpansionServiceProcess() error {
args := []string{"-m", expansionServiceEntrypoint, "-p",
strconv.Itoa(*port), "--fully_qualified_name_glob", "*"}
- if *requirements_file != "" {
- log.Printf("Received the requirements file %v",
*requirements_file)
- updatedRequirementsFileName, err :=
getUpdatedRequirementsFile(*requirements_file, *dependencies_dir)
+ // Requirements file with dependencies to install.
+ // Note that we have to look for the requirements file in the
dependencies
+ // volume here not the requirements file at the top level. Latter
provides
+
(beam) branch asf-site updated: Publishing website 2024/03/12 23:38:13 at commit 5a9a7d2
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 784f2ee25fe Publishing website 2024/03/12 23:38:13 at commit 5a9a7d2 784f2ee25fe is described below commit 784f2ee25fe4488a33afe91a98a580e9c018f2fc Author: runner AuthorDate: Tue Mar 12 23:38:13 2024 + Publishing website 2024/03/12 23:38:13 at commit 5a9a7d2 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index d84a1fa69b9..1a0e1cd181e 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.54.0/2024-03-12T12:21:53-04:00/categories/blog/2024-03-12T12:21:53-04:00/blog/2024-03-12T12:21:53-04:00/categories/2024-03-12T12:21:53-04:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.54.0/2024-03-12T19:04:55-04:00/categories/blog/2024-03-12T19:04:55-04:00/blog/2024-03-12T19:04:55-04:00/categories/2024-03-12T19:04:55-04:00/catego [...] \ No newline at end of file
(beam) branch master updated: Add a known issue for gcsio in the notes of 2.53.0 and 2.54.0. (#30615)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 5a9a7d2f85b Add a known issue for gcsio in the notes of 2.53.0 and 2.54.0. (#30615) 5a9a7d2f85b is described below commit 5a9a7d2f85ba393de7c01ac750b18df95db00323 Author: Shunping Huang AuthorDate: Tue Mar 12 19:04:55 2024 -0400 Add a known issue for gcsio in the notes of 2.53.0 and 2.54.0. (#30615) --- CHANGES.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CHANGES.md b/CHANGES.md index f1c94a40af3..9644578091f 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -98,6 +98,7 @@ * Fixed X (Java/Python) ([#X](https://github.com/apache/beam/issues/X)). * Fixed SpannerIO.readChangeStream to support propagating credentials from pipeline options to the getDialect calls for authenticating with Spanner (Java) ([#30361](https://github.com/apache/beam/pull/30361)). +* Reduced the number of HTTP requests in GCSIO function calls (Python) ([#30205](https://github.com/apache/beam/pull/30205)) ## Security Fixes * Fixed (CVE--)[https://www.cve.org/CVERecord?id=CVE--] (Java/Python/Go) ([#X](https://github.com/apache/beam/issues/X)). @@ -145,6 +146,7 @@ ## Known Issues * Some Python pipelines that run with 2.52.0-2.54.0 SDKs and use large materialized side inputs might be affected by a performance regression. To restore the prior behavior on these SDK versions, supply the `--max_cache_memory_usage_mb=0` pipeline option. ([#30360](https://github.com/apache/beam/issues/30360)). +* Python pipelines that run with 2.53.0-2.54.0 SDKs and perform file operations on GCS might be affected by excess HTTP requests. This could lead to a performance regression or a permission issue. ([#28398](https://github.com/apache/beam/issues/28398)) # [2.53.0] - 2024-01-04 @@ -187,6 +189,7 @@ * Potential race condition causing NPE in DataflowExecutionStateSampler in Dataflow Java Streaming pipelines ([#29987](https://github.com/apache/beam/issues/29987)). * Some Python pipelines that run with 2.52.0-2.54.0 SDKs and use large materialized side inputs might be affected by a performance regression. To restore the prior behavior on these SDK versions, supply the `--max_cache_memory_usage_mb=0` pipeline option. ([#30360](https://github.com/apache/beam/issues/30360)). +* Python pipelines that run with 2.53.0-2.54.0 SDKs and perform file operations on GCS might be affected by excess HTTP requests. This could lead to a performance regression or a permission issue. ([#28398](https://github.com/apache/beam/issues/28398)) # [2.52.0] - 2023-11-17
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 83712d1717b Updating config from bot
83712d1717b is described below
commit 83712d1717badbe44eae67dfd82eaf7e0a2a04a8
Author: github-actions
AuthorDate: Tue Mar 12 21:06:21 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 2c27790683b..2a7de29c074 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1710265960889,
-"Abacn": 1709822751103
+"Abacn": 1710277577945
}
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new ea1022565b5 Updating config from bot
ea1022565b5 is described below
commit ea1022565b51a3837f6501213b3c335783fcd62c
Author: github-actions
AuthorDate: Tue Mar 12 21:06:16 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
index ab7785418d6..ba1618efdee 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
@@ -3,7 +3,7 @@
"dateOfLastReviewAssignment": {
"Abacn": 1710034125002,
"AnandInguva": 1710144575340,
-"chamikaramj": 1709314689551,
+"chamikaramj": 1710277574566,
"damccorm": 1709429281257,
"johnjcasey": 1709562907440,
"jrmccluskey": 1709647558176,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f8038784f57 Updating config from bot
f8038784f57 is described below
commit f8038784f57c946262c8c45ea2e4129a00ac7fbb
Author: github-actions
AuthorDate: Tue Mar 12 21:06:18 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-30614.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30614.json
b/scripts/ci/pr-bot/state/pr-state/pr-30614.json
new file mode 100644
index 000..595c244fe1b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30614.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "tvalentyn",
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 2bfa7f490eb Updating config from bot 2bfa7f490eb is described below commit 2bfa7f490eb9cc7e7caff66a2bc5f4c804960c27 Author: github-actions AuthorDate: Tue Mar 12 21:06:20 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index dabe984c7b8..ed651ca379d 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -5,7 +5,7 @@ "yeandy": 1665802753763, "TheNeuralBit": 1667896849319, "ryanthompson591": 1670002443548, -"tvalentyn": 1709584498255, +"tvalentyn": 1710277577945, "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1709863822385,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 82191be306b Updating config from bot 82191be306b is described below commit 82191be306be5aa0e2775d2bf29d24193bf7cc0f Author: github-actions AuthorDate: Tue Mar 12 21:06:22 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30605.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30605.json b/scripts/ci/pr-bot/state/pr-state/pr-30605.json index 3077b305e87..c756775542a 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-30605.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-30605.json @@ -8,5 +8,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 50ed3e3e466 Updating config from bot
50ed3e3e466 is described below
commit 50ed3e3e466f75732af0613bca104dba7608bce8
Author: github-actions
AuthorDate: Tue Mar 12 21:06:15 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-30615.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30615.json
b/scripts/ci/pr-bot/state/pr-state/pr-30615.json
new file mode 100644
index 000..26e4be5dc8a
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30615.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "chamikaramj"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new b0035b341ed Updating config from bot b0035b341ed is described below commit b0035b341edb8e78f6e0d633628ef23945c994db Author: github-actions AuthorDate: Tue Mar 12 17:52:43 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 7f0dcd420cb..dabe984c7b8 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -11,6 +11,7 @@ "damccorm": 1709863822385, "jrmccluskey": 1709598146839, "riteshghorse": 1709764491759, -"liferoad": 1709786079956 +"liferoad": 1709786079956, +"shunping": 1710265960889 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 1d741d38f5e Updating config from bot 1d741d38f5e is described below commit 1d741d38f5e36269dbc4ce1b389f8c4249a4cf4c Author: github-actions AuthorDate: Tue Mar 12 17:52:45 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index 49e16f49ebd..85f59212303 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -8,6 +8,7 @@ "ahmedabu98": 1709760886425, "bvolpato": 1709228061783, "manavgarg": 1690826779210, -"damondouglas": 1709598146839 +"damondouglas": 1709598146839, +"shunping": 1710265960889 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new d99b4b6ada8 Updating config from bot
d99b4b6ada8 is described below
commit d99b4b6ada8b3378c521636dd12186afe6ff10c1
Author: github-actions
AuthorDate: Tue Mar 12 17:52:44 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index f7da2c3c233..2c27790683b 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1709667439988,
+"damccorm": 1710265960889,
"Abacn": 1709822751103
}
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 6558e532e13 Updating config from bot
6558e532e13 is described below
commit 6558e532e139e3bbfc5a7180ce4c38e32de588bd
Author: github-actions
AuthorDate: Tue Mar 12 17:52:42 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-30605.json | 8 ++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30605.json
b/scripts/ci/pr-bot/state/pr-state/pr-30605.json
index 242a48d7d3b..3077b305e87 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-30605.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30605.json
@@ -1,7 +1,11 @@
{
"commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "reviewersAssignedForLabels": {
+"python": "shunping",
+"build": "damccorm",
+"io": "shunping"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
(beam) branch users/damccorm/blog-template deleted (was 35f0fd2db66)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/blog-template in repository https://gitbox.apache.org/repos/asf/beam.git was 35f0fd2db66 Update blog template with known issues link to CHANGES The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated: Update blog template with known issues link to CHANGES (#30611)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 986f67fd92b Update blog template with known issues link to CHANGES
(#30611)
986f67fd92b is described below
commit 986f67fd92b27dee0c95c5965de98ba6f84a0143
Author: Danny McCormick
AuthorDate: Tue Mar 12 12:21:53 2024 -0400
Update blog template with known issues link to CHANGES (#30611)
---
contributor-docs/release-guide.md | 2 ++
1 file changed, 2 insertions(+)
diff --git a/contributor-docs/release-guide.md
b/contributor-docs/release-guide.md
index 559d3c0f9f0..e1d0c67e049 100644
--- a/contributor-docs/release-guide.md
+++ b/contributor-docs/release-guide.md
@@ -742,6 +742,8 @@ as an example.
* {$KNOWN_ISSUE_1}
* {$KNOWN_ISSUE_2}
+For the most up to date list of known issues, see
https://github.com/apache/beam/blob/master/CHANGES.md
+
## List of Contributors
According to git shortlog, the following people contributed to the
{$RELEASE_VERSION} release. Thank you to all contributors!
(beam) branch master updated (0c8b78a9ff0 -> 9a6777ebe7e)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 0c8b78a9ff0 Remove Anand, add Shunping to reviewers (#30608) add 97f8dae92f2 Put an explanatory comment in trigger files that exist new 9a6777ebe7e Merge pull request #30586: Put an explanatory comment in trigger files that exist The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/trigger_files/beam_PostCommit_Java_Avro_Versions.json | 3 +++ .github/trigger_files/beam_PostCommit_Java_DataflowV1.json| 3 +++ .github/trigger_files/beam_PostCommit_Java_PVR_Spark3_Streaming.json | 3 +++ .github/trigger_files/beam_PostCommit_Java_PVR_Spark_Batch.json | 3 +++ .../beam_PostCommit_Java_ValidatesRunner_Dataflow_V2.json | 3 +++ .../beam_PostCommit_Java_ValidatesRunner_Dataflow_V2_Streaming.json | 3 +++ .../beam_PostCommit_Java_ValidatesRunner_Flink_Java11.json| 3 +++ ...beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming.json | 3 +++ .../trigger_files/beam_PostCommit_Java_ValidatesRunner_Twister2.json | 3 +++ .github/trigger_files/beam_PostCommit_Java_ValidatesRunner_ULR.json | 3 +++ .github/trigger_files/beam_PostCommit_Python_Examples_Direct.json | 3 +++ .github/trigger_files/beam_PostCommit_Python_Xlang_Gcp_Dataflow.json | 3 +++ .github/trigger_files/beam_PostCommit_Python_Xlang_Gcp_Direct.json| 4 +++- .github/trigger_files/beam_PostCommit_Python_Xlang_IO_Dataflow.json | 3 +++ .github/trigger_files/beam_PostCommit_XVR_Flink.json | 3 +++ .../beam_PostCommit_XVR_PythonUsingJavaSQL_Dataflow.json | 3 +++ 16 files changed, 48 insertions(+), 1 deletion(-) create mode 100644 .github/trigger_files/beam_PostCommit_XVR_Flink.json
(beam) 01/01: Merge pull request #30586: Put an explanatory comment in trigger files that exist
This is an automated email from the ASF dual-hosted git repository. kenn pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 9a6777ebe7e0903f9ffb57f7ea9e84bb4e448170 Merge: 0c8b78a9ff0 97f8dae92f2 Author: Kenneth Knowles AuthorDate: Tue Mar 12 11:59:47 2024 -0400 Merge pull request #30586: Put an explanatory comment in trigger files that exist .github/trigger_files/beam_PostCommit_Java_Avro_Versions.json | 3 +++ .github/trigger_files/beam_PostCommit_Java_DataflowV1.json| 3 +++ .github/trigger_files/beam_PostCommit_Java_PVR_Spark3_Streaming.json | 3 +++ .github/trigger_files/beam_PostCommit_Java_PVR_Spark_Batch.json | 3 +++ .../beam_PostCommit_Java_ValidatesRunner_Dataflow_V2.json | 3 +++ .../beam_PostCommit_Java_ValidatesRunner_Dataflow_V2_Streaming.json | 3 +++ .../beam_PostCommit_Java_ValidatesRunner_Flink_Java11.json| 3 +++ ...beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming.json | 3 +++ .../trigger_files/beam_PostCommit_Java_ValidatesRunner_Twister2.json | 3 +++ .github/trigger_files/beam_PostCommit_Java_ValidatesRunner_ULR.json | 3 +++ .github/trigger_files/beam_PostCommit_Python_Examples_Direct.json | 3 +++ .github/trigger_files/beam_PostCommit_Python_Xlang_Gcp_Dataflow.json | 3 +++ .github/trigger_files/beam_PostCommit_Python_Xlang_Gcp_Direct.json| 4 +++- .github/trigger_files/beam_PostCommit_Python_Xlang_IO_Dataflow.json | 3 +++ .github/trigger_files/beam_PostCommit_XVR_Flink.json | 3 +++ .../beam_PostCommit_XVR_PythonUsingJavaSQL_Dataflow.json | 3 +++ 16 files changed, 48 insertions(+), 1 deletion(-)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 251a882831b Updating config from bot
251a882831b is described below
commit 251a882831b9602864b6b0e4b10247e495bc20c4
Author: github-actions
AuthorDate: Tue Mar 12 15:38:41 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-30611.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30611.json
b/scripts/ci/pr-bot/state/pr-state/pr-30611.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30611.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch users/damccorm/blog-template created (now 35f0fd2db66)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/blog-template in repository https://gitbox.apache.org/repos/asf/beam.git at 35f0fd2db66 Update blog template with known issues link to CHANGES This branch includes the following new commits: new 35f0fd2db66 Update blog template with known issues link to CHANGES The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) 01/01: Update blog template with known issues link to CHANGES
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/blog-template
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 35f0fd2db66c143474cfdad603b091a5dd9f2804
Author: Danny McCormick
AuthorDate: Tue Mar 12 11:29:25 2024 -0400
Update blog template with known issues link to CHANGES
---
contributor-docs/release-guide.md | 2 ++
1 file changed, 2 insertions(+)
diff --git a/contributor-docs/release-guide.md
b/contributor-docs/release-guide.md
index 559d3c0f9f0..e1d0c67e049 100644
--- a/contributor-docs/release-guide.md
+++ b/contributor-docs/release-guide.md
@@ -742,6 +742,8 @@ as an example.
* {$KNOWN_ISSUE_1}
* {$KNOWN_ISSUE_2}
+For the most up to date list of known issues, see
https://github.com/apache/beam/blob/master/CHANGES.md
+
## List of Contributors
According to git shortlog, the following people contributed to the
{$RELEASE_VERSION} release. Thank you to all contributors!
(beam) branch users/damccorm/reviewers deleted (was ae6c84c257a)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/reviewers in repository https://gitbox.apache.org/repos/asf/beam.git was ae6c84c257a Add shunping to fallback reviewers The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated: Remove Anand, add Shunping to reviewers (#30608)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 0c8b78a9ff0 Remove Anand, add Shunping to reviewers (#30608) 0c8b78a9ff0 is described below commit 0c8b78a9ff0b393a89a40b4e389dec25d1b319e5 Author: Danny McCormick AuthorDate: Tue Mar 12 10:52:57 2024 -0400 Remove Anand, add Shunping to reviewers (#30608) * Remove Anand, add Shunping to reviewers * Add shunping to fallback reviewers --- .github/REVIEWERS.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/REVIEWERS.yml b/.github/REVIEWERS.yml index b878d2f81cc..44e80bd594a 100644 --- a/.github/REVIEWERS.yml +++ b/.github/REVIEWERS.yml @@ -27,12 +27,12 @@ labels: - youngoli - name: Python reviewers: - - AnandInguva - damccorm - jrmccluskey - tvalentyn - riteshghorse - liferoad + - shunping exclusionList: [] - name: Java reviewers: @@ -51,6 +51,7 @@ labels: - ahmedabu98 - bvolpato - damondouglas + - shunping exclusionList: [] - name: spanner reviewers: @@ -79,7 +80,6 @@ labels: exclusionList: [] fallbackReviewers: - Abacn - - AnandInguva - chamikaramj - damccorm - damondouglas @@ -89,4 +89,5 @@ fallbackReviewers: - lostluck - riteshghorse - robertwb + - shunping - tvalentyn
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 67154d1fa77 Updating config from bot
67154d1fa77 is described below
commit 67154d1fa7782995aa2371f1e72fd6bc2c90e331
Author: github-actions
AuthorDate: Tue Mar 12 14:25:33 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-30608.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30608.json
b/scripts/ci/pr-bot/state/pr-state/pr-30608.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30608.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch users/damccorm/reviewers updated (56f69dcb1f1 -> ae6c84c257a)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/reviewers in repository https://gitbox.apache.org/repos/asf/beam.git from 56f69dcb1f1 Remove Anand, add Shunping to reviewers add ae6c84c257a Add shunping to fallback reviewers No new revisions were added by this update. Summary of changes: .github/REVIEWERS.yml | 1 + 1 file changed, 1 insertion(+)
(beam) branch users/damccorm/reviewers created (now 56f69dcb1f1)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/reviewers in repository https://gitbox.apache.org/repos/asf/beam.git at 56f69dcb1f1 Remove Anand, add Shunping to reviewers This branch includes the following new commits: new 56f69dcb1f1 Remove Anand, add Shunping to reviewers The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) 01/01: Remove Anand, add Shunping to reviewers
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a commit to branch users/damccorm/reviewers in repository https://gitbox.apache.org/repos/asf/beam.git commit 56f69dcb1f1d97eeda9b93fcd4f4131f98e089d6 Author: Danny McCormick AuthorDate: Tue Mar 12 10:23:24 2024 -0400 Remove Anand, add Shunping to reviewers --- .github/REVIEWERS.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/REVIEWERS.yml b/.github/REVIEWERS.yml index b878d2f81cc..8a8c3b84af4 100644 --- a/.github/REVIEWERS.yml +++ b/.github/REVIEWERS.yml @@ -27,12 +27,12 @@ labels: - youngoli - name: Python reviewers: - - AnandInguva - damccorm - jrmccluskey - tvalentyn - riteshghorse - liferoad + - shunping exclusionList: [] - name: Java reviewers: @@ -51,6 +51,7 @@ labels: - ahmedabu98 - bvolpato - damondouglas + - shunping exclusionList: [] - name: spanner reviewers: @@ -79,7 +80,6 @@ labels: exclusionList: [] fallbackReviewers: - Abacn - - AnandInguva - chamikaramj - damccorm - damondouglas
(beam) branch master updated: Duet AI documentation lookup prompts for Beam ML (#30351)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 35252b92c0f Duet AI documentation lookup prompts for Beam ML (#30351) 35252b92c0f is described below commit 35252b92c0f88a13251c92f5f5c5799499d1f2a2 Author: Oleh Borysevych AuthorDate: Tue Mar 12 14:52:09 2024 +0100 Duet AI documentation lookup prompts for Beam ML (#30351) * ml prompts initial commit * preprocess data added * orchedstration prompts added * orchestration prompts * new files added * ml prompts added * bigqueryml prompt added * fix whitespace * add model refresh prompt * custom inference added * Apply suggestions from code review Co-authored-by: Daria Bezkorovaina <99457894+dariabezkorova...@users.noreply.github.com> * llm inference added * whitespace fixed * code review comments * nolinks docs added * Apply suggestions from code review Co-authored-by: Daria Bezkorovaina <99457894+dariabezkorova...@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Daria Bezkorovaina <99457894+dariabezkorova...@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Daria Bezkorovaina <99457894+dariabezkorova...@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Daria Bezkorovaina <99457894+dariabezkorova...@users.noreply.github.com> * fixed broken link * Apply suggestions from code review Co-authored-by: Daria Bezkorovaina <99457894+dariabezkorova...@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Daria Bezkorovaina <99457894+dariabezkorova...@users.noreply.github.com> * code review comments * whitespace fix * Apply suggestions from code review Co-authored-by: Danny McCormick - Co-authored-by: Daria Bezkorovaina <99457894+dariabezkorova...@users.noreply.github.com> Co-authored-by: Danny McCormick --- .../33_ai_ml_pipelines.md | 22 .../36_ai_kfp_orchestration.md | 51 ++ .../37_ai_tfx_orchestration.md | 20 .../38_ai_runinference.md | 37 + .../39_ai_runinference_metrics.md | 38 ++ .../40_ai_bigquery_ml.md | 48 + .../41_ai_model_refresh.md | 42 +++ .../42_ai_custom_inference.md | 50 ++ .../43_ai_llm_inference.md | 60 ++ .../documentation-lookup/33_ai_ml_pipelines.md | 15 ++ .../36_ai_kfp_orchestration.md | 51 ++ .../37_ai_tfx_orchestration.md | 20 .../documentation-lookup/38_ai_runinference.md | 39 ++ .../39_ai_runinference_metrics.md | 39 ++ .../documentation-lookup/40_ai_bigquery_ml.md | 52 +++ .../documentation-lookup/41_ai_model_refresh.md| 42 +++ .../documentation-lookup/42_ai_custom_inference.md | 50 ++ .../documentation-lookup/43_ai_llm_inference.md| 60 ++ 18 files changed, 736 insertions(+) diff --git a/learning/prompts/documentation-lookup-nolinks/33_ai_ml_pipelines.md b/learning/prompts/documentation-lookup-nolinks/33_ai_ml_pipelines.md new file mode 100644 index 000..30d62c0faac --- /dev/null +++ b/learning/prompts/documentation-lookup-nolinks/33_ai_ml_pipelines.md @@ -0,0 +1,22 @@ +Prompt: +How can I use Apache Beam in my AI/ML pipeline? + +Response: +Apache Beam is an open-source, unified programming model for defining batch and streaming data-parallel processing pipelines. Each Apache Beam pipeline is represented as a Directed Acyclic Graph (DAG) of data processing operations constructed using the Apache Beam SDK in one of the supported languages. To execute a pipeline, you need to deploy it to one of the supported Beam runners. + +You can use Apache Beam for various tasks within your AI/ML pipeline, including data validation, preprocessing, model validation, and model deployment and inference. + +Apache Beam offers a rich set of I/O connectors and `PTransforms` that allow for reading and writing data from and to various data sources and sinks, as well as performing data validation. + +For data preprocessing, Apache Beam provides the `MLTransform` class. This feature allows you to ensure data consistency by applying the same preprocessing steps for both tra
