(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 28580c660d1 Updating config from bot
28580c660d1 is described below
commit 28580c660d1773d53886f5a2cc9f290f683eb443
Author: github-actions
AuthorDate: Wed May 1 05:05:42 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31144.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31144.json
b/scripts/ci/pr-bot/state/pr-state/pr-31144.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31144.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch dependabot/go_modules/sdks/github.com/docker/docker-26.1.0incompatible deleted (was 9e7fcab7396)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/docker/docker-26.1.0incompatible in repository https://gitbox.apache.org/repos/asf/beam.git was 9e7fcab7396 Bump github.com/docker/docker in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch dependabot/go_modules/sdks/github.com/docker/docker-26.1.1incompatible created (now 440444008d0)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/docker/docker-26.1.1incompatible in repository https://gitbox.apache.org/repos/asf/beam.git at 440444008d0 Bump github.com/docker/docker in /sdks No new revisions were added by this update.
(beam) branch nightly-refs/heads/master updated (970109e0553 -> 43bf99ddcef)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 970109e0553 Also allow links to Drive materials. (#31131)
add f0d06051072 add finalizers to kafka loadbalancer services (#31130)
add 413af128937 Allow users to configure wait options for new Neo4j
databases (#31129)
add 22535521e7b Bump com.gradle.enterprise from 3.15.1 to 3.17.2
add e3b4d41f6b8 Merge pull request #31032: Bump com.gradle.enterprise from
3.15.1 to 3.17.2
add 5f71e6a592c remove atomic and wait from strimzi helm terrafrom (#31132)
add 9612fe13eaf Elasticsearch enable Point In Time based searches (#30824)
add 356311fb75d Fix typo (#31063)
add 4a134b5f002 Enable BigQueryMetrics by default (#31015)
add 7f7bc3e9ce5 Optimise View.asList() side inputs for iterating rather
than for indexing. (#31087)
add 43bf99ddcef Update game_stats.py (#31134)
No new revisions were added by this update.
Summary of changes:
.../kafka/strimzi/01-strimzi-operator/kafka.tf | 3 +-
.../gke-internal-load-balanced/listeners.yaml | 3 +
CHANGES.md | 5 +
.../{package-info.java => DatabaseWaitOption.java} | 4 +-
.../apache/beam/it/neo4j/DatabaseWaitOptions.java | 68
.../apache/beam/it/neo4j/Neo4jResourceManager.java | 41 ++-
.../beam/it/neo4j/Neo4jResourceManagerIT.java | 2 +-
.../beam/it/neo4j/Neo4jResourceManagerTest.java| 15 +-
.../dataflow/worker/StreamingDataflowWorker.java | 2 +-
.../apache/beam/sdk/options/StreamingOptions.java | 20 +
.../java/org/apache/beam/sdk/transforms/View.java | 57 ++-
.../apache/beam/sdk/values/PCollectionViews.java | 210 ++-
.../org/apache/beam/sdk/transforms/ViewTest.java | 33 ++
.../CreatePCollectionViewTranslationTest.java | 2 +-
.../sdk/util/construction/MorePipelineTest.java| 2 +-
.../sdk/io/elasticsearch/ElasticsearchIOIT.java| 6 +
.../sdk/io/elasticsearch/ElasticsearchIOTest.java | 20 +-
.../elasticsearch/ElasticsearchIOTestCommon.java | 31 +-
.../io/elasticsearch/ElasticsearchIOTestUtils.java | 7 +-
.../beam/sdk/io/elasticsearch/ElasticsearchIO.java | 401 +
.../sdk/io/gcp/bigquery/BigQuerySinkMetrics.java | 2 +-
.../examples/complete/game/game_stats.py | 2 +-
settings.gradle.kts| 2 +-
23 files changed, 842 insertions(+), 96 deletions(-)
copy it/neo4j/src/main/java/org/apache/beam/it/neo4j/{package-info.java =>
DatabaseWaitOption.java} (92%)
create mode 100644
it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOptions.java
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 10c5bbed739 Updating config from bot
10c5bbed739 is described below
commit 10c5bbed739804d085db93ee645510864cf1a65a
Author: github-actions
AuthorDate: Wed May 1 01:31:42 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31141.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31141.json
b/scripts/ci/pr-bot/state/pr-state/pr-31141.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31141.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new bdea032f139 Updating config from bot
bdea032f139 is described below
commit bdea032f139cba69b2b2f552ca6696f6a16cf83c
Author: github-actions
AuthorDate: Wed May 1 01:31:44 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31140.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31140.json
b/scripts/ci/pr-bot/state/pr-state/pr-31140.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31140.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 207d797ac17 Updating config from bot
207d797ac17 is described below
commit 207d797ac17e29d6dabfe46c79279273b4dd0e3f
Author: github-actions
AuthorDate: Wed May 1 01:31:39 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 6ca0fb07fda..d8ebbef1470 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1714511219401,
-"Abacn": 1714466105691
+"Abacn": 1714527097428
}
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 21167fc4621 Updating config from bot
21167fc4621 is described below
commit 21167fc462182efb3a339f00eae1c8d99a486474
Author: github-actions
AuthorDate: Wed May 1 01:31:38 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31143.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31143.json
b/scripts/ci/pr-bot/state/pr-state/pr-31143.json
new file mode 100644
index 000..9c51466b9ee
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31143.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e8c91324d94 Updating config from bot
e8c91324d94 is described below
commit e8c91324d9442ae343a8609c6287cd2781117ff2
Author: github-actions
AuthorDate: Wed May 1 00:21:24 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31137.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31137.json
b/scripts/ci/pr-bot/state/pr-state/pr-31137.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31137.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 5e19f24b300 Updating config from bot 5e19f24b300 is described below commit 5e19f24b300a84387d3eb72e54e5d539caa712a6 Author: github-actions AuthorDate: Wed May 1 00:21:20 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index 01919e5523b..9f481f34a2e 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -13,6 +13,6 @@ "robertwb": 1714025210513, "tvalentyn": 1714408536058, "damondouglas": 1713917985911, -"shunping": 1713261957540 +"shunping": 1714522877825 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e9960889286 Updating config from bot
e9960889286 is described below
commit e9960889286b00e095fc2cf0ab0e0b71b7d79ff7
Author: github-actions
AuthorDate: Wed May 1 00:21:19 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31142.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31142.json
b/scripts/ci/pr-bot/state/pr-state/pr-31142.json
new file mode 100644
index 000..337ace80587
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31142.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "shunping"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 32620452356 Updating config from bot
32620452356 is described below
commit 32620452356e89404fb3059e3c908a629880d173
Author: github-actions
AuthorDate: Tue Apr 30 23:34:42 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31136.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31136.json
b/scripts/ci/pr-bot/state/pr-state/pr-31136.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31136.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch master updated: Update game_stats.py (#31134)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 43bf99ddcef Update game_stats.py (#31134) 43bf99ddcef is described below commit 43bf99ddcefbb0eac0b8ecbe2eaf99d409719a99 Author: Ahmet Altay AuthorDate: Tue Apr 30 15:31:03 2024 -0700 Update game_stats.py (#31134) Fixing a typo in examples & docs. --- sdks/python/apache_beam/examples/complete/game/game_stats.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdks/python/apache_beam/examples/complete/game/game_stats.py b/sdks/python/apache_beam/examples/complete/game/game_stats.py index d6f5aab3e7b..233d22b7542 100644 --- a/sdks/python/apache_beam/examples/complete/game/game_stats.py +++ b/sdks/python/apache_beam/examples/complete/game/game_stats.py @@ -196,7 +196,7 @@ class WriteToBigQuery(beam.PTransform): # [START abuse_detect] class CalculateSpammyUsers(beam.PTransform): """Filter out all but those users with a high clickrate, which we will - consider as 'spammy' uesrs. + consider as 'spammy' users. We do this by finding the mean total score per user, then using that information as a side input to filter out all but those user scores that are
(beam) branch aaltay-patch-1 deleted (was 1bb3931eb6b)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch aaltay-patch-1 in repository https://gitbox.apache.org/repos/asf/beam.git was 1bb3931eb6b Update game_stats.py The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 9e5bb093fa4 Updating config from bot
9e5bb093fa4 is described below
commit 9e5bb093fa41e052af6c27937098d77320e14628
Author: github-actions
AuthorDate: Tue Apr 30 21:07:06 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31134.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31134.json
b/scripts/ci/pr-bot/state/pr-state/pr-31134.json
new file mode 100644
index 000..321b31c0f05
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31134.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "tvalentyn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f311778e375 Updating config from bot
f311778e375 is described below
commit f311778e375f9a838c01227f97ec92c1be852d48
Author: github-actions
AuthorDate: Tue Apr 30 21:07:03 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 6d62c141f88..6ca0fb07fda 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1714135257527,
+"damccorm": 1714511219401,
"Abacn": 1714466105691
}
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 6c6390b39cf Updating config from bot 6c6390b39cf is described below commit 6c6390b39cfc52d2f7ac3e52d3a8d17f830b3d65 Author: github-actions AuthorDate: Tue Apr 30 21:07:08 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index ccee0c88527..8c4fb082151 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -5,7 +5,7 @@ "yeandy": 1665802753763, "TheNeuralBit": 1667896849319, "ryanthompson591": 1670002443548, -"tvalentyn": 1713389714559, +"tvalentyn": 1714511224768, "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1713528836263,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e96e65cab3e Updating config from bot
e96e65cab3e is described below
commit e96e65cab3efeca6f7694d1c790819444b3a542d
Author: github-actions
AuthorDate: Tue Apr 30 21:07:01 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31135.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31135.json
b/scripts/ci/pr-bot/state/pr-state/pr-31135.json
new file mode 100644
index 000..8a92900947e
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31135.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch master updated: Optimise View.asList() side inputs for iterating rather than for indexing. (#31087)
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 7f7bc3e9ce5 Optimise View.asList() side inputs for iterating rather
than for indexing. (#31087)
7f7bc3e9ce5 is described below
commit 7f7bc3e9ce54dc6f732e20bef008ecdef571033e
Author: Robert Bradshaw
AuthorDate: Tue Apr 30 13:58:03 2024 -0700
Optimise View.asList() side inputs for iterating rather than for indexing.
(#31087)
The current implementation is, essentially, a distributed hashmap from
integer keys to the list contents, mediated by each upstream worker starting
at a random value to minimize overlaps and emitting sufficient metadata to
map
this onto the contiguous range [0, N). This provides optimal *random-access*
performance, but very poor *iteration* performance (essentially having to do
a key lookup for every advance, and as the keys are hashed and distributed
rather than clustered numerically, there is little to no amortization in
these
lookups for adjacent items.
Given that most uses for List side inputs are merely to gather a collection
of values (the user has no control over the ordering when materialized) and
the high costs of providing random access, this is probably the wrong
tradeoff
for most pipelines.
This is an update-incompatible change and so has been guarded by the
update compatibility version flag. The old behavior can be explicitly
asked for via a new AsList#withRandomAccess() method.
---
CHANGES.md | 5 +
.../apache/beam/sdk/options/StreamingOptions.java | 20 ++
.../java/org/apache/beam/sdk/transforms/View.java | 57 +-
.../apache/beam/sdk/values/PCollectionViews.java | 210 -
.../org/apache/beam/sdk/transforms/ViewTest.java | 33
.../CreatePCollectionViewTranslationTest.java | 2 +-
.../sdk/util/construction/MorePipelineTest.java| 2 +-
7 files changed, 323 insertions(+), 6 deletions(-)
diff --git a/CHANGES.md b/CHANGES.md
index ec7fbe45668..66e40126236 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -71,6 +71,11 @@
## Breaking Changes
* X behavior was changed ([#X](https://github.com/apache/beam/issues/X)).
+* Java's View.asList() side inputs are now optimized for iterating rather than
+ indexing when in the global window.
+ This new implementation still supports all (immutable) List methods as
before,
+ but some of the random access methods like get() and size() will be slower.
+ To use the old implementation one can use View.asList().withRandomAccess().
## Deprecations
diff --git
a/sdks/java/core/src/main/java/org/apache/beam/sdk/options/StreamingOptions.java
b/sdks/java/core/src/main/java/org/apache/beam/sdk/options/StreamingOptions.java
index e389ab89cf9..8065e0a40cb 100644
---
a/sdks/java/core/src/main/java/org/apache/beam/sdk/options/StreamingOptions.java
+++
b/sdks/java/core/src/main/java/org/apache/beam/sdk/options/StreamingOptions.java
@@ -17,6 +17,10 @@
*/
package org.apache.beam.sdk.options;
+import java.util.Arrays;
+import java.util.Comparator;
+import java.util.List;
+import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.Comparators;
import org.checkerframework.checker.nullness.qual.Nullable;
/** Options used to configure streaming. */
@@ -41,4 +45,20 @@ public interface StreamingOptions extends
ApplicationNameOptions, PipelineOption
String getUpdateCompatibilityVersion();
void setUpdateCompatibilityVersion(@Nullable String
updateCompatibilityVersion);
+
+ static boolean updateCompatibilityVersionLessThan(PipelineOptions options,
String version) {
+if (options == null) {
+ return false;
+}
+String updateCompatibilityVersion =
+options.as(StreamingOptions.class).getUpdateCompatibilityVersion();
+if (updateCompatibilityVersion == null) {
+ return false;
+}
+List requestedVersion =
Arrays.asList(updateCompatibilityVersion.split("\\."));
+List targetVersion = Arrays.asList(version.split("\\."));
+return Comparators.lexicographical(Comparator.naturalOrder())
+.compare(requestedVersion, targetVersion)
+< 0;
+ }
}
diff --git
a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/View.java
b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/View.java
index 22c42249678..ca04542b372 100644
--- a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/View.java
+++ b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/View.java
@@ -28,8 +28,10 @@ import org.apache.beam.sdk.coders.Coder;
import org.apache.beam.sdk.coders.CoderException;
import org.apache.beam.sdk.coders.KvCoder;
import org.apache.beam.sdk.io.range.OffsetRange;
+import org.apache.beam.sdk.options.StreamingOptions;
import
(beam) branch aaltay-patch-1 created (now 1bb3931eb6b)
This is an automated email from the ASF dual-hosted git repository. altay pushed a change to branch aaltay-patch-1 in repository https://gitbox.apache.org/repos/asf/beam.git at 1bb3931eb6b Update game_stats.py This branch includes the following new commits: new 1bb3931eb6b Update game_stats.py The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) 01/01: Update game_stats.py
This is an automated email from the ASF dual-hosted git repository. altay pushed a commit to branch aaltay-patch-1 in repository https://gitbox.apache.org/repos/asf/beam.git commit 1bb3931eb6b147bc7c3f2dabf32e8492e7de617c Author: Ahmet Altay AuthorDate: Tue Apr 30 13:13:26 2024 -0700 Update game_stats.py Fixing a typo in examples & docs. --- sdks/python/apache_beam/examples/complete/game/game_stats.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdks/python/apache_beam/examples/complete/game/game_stats.py b/sdks/python/apache_beam/examples/complete/game/game_stats.py index d6f5aab3e7b..233d22b7542 100644 --- a/sdks/python/apache_beam/examples/complete/game/game_stats.py +++ b/sdks/python/apache_beam/examples/complete/game/game_stats.py @@ -196,7 +196,7 @@ class WriteToBigQuery(beam.PTransform): # [START abuse_detect] class CalculateSpammyUsers(beam.PTransform): """Filter out all but those users with a high clickrate, which we will - consider as 'spammy' uesrs. + consider as 'spammy' users. We do this by finding the mean total score per user, then using that information as a side input to filter out all but those user scores that are
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 93e5efb3cae Updating config from bot 93e5efb3cae is described below commit 93e5efb3caeb41a46239412c838cea2bbcdb Author: github-actions AuthorDate: Tue Apr 30 20:02:08 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-31125.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31125.json b/scripts/ci/pr-bot/state/pr-state/pr-31125.json index 613bd2cda0a..b56df7de73f 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-31125.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-31125.json @@ -5,7 +5,7 @@ "io": "chamikaramj" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
(beam) branch master updated: Enable BigQueryMetrics by default (#31015)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 4a134b5f002 Enable BigQueryMetrics by default (#31015)
4a134b5f002 is described below
commit 4a134b5f002b2d8bf0101eef5b41870f813f8fee
Author: JayajP
AuthorDate: Tue Apr 30 12:33:06 2024 -0700
Enable BigQueryMetrics by default (#31015)
---
.../apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java| 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
index 82a9ff15b6b..660680cda3e 100644
---
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
+++
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
@@ -612,7 +612,7 @@ public class StreamingDataflowWorker {
MetricsEnvironment.setProcessWideContainer(new MetricsLogger(null));
if (options.isEnableStreamingEngine()
-&& DataflowRunner.hasExperiment(options, "enable_per_worker_metrics"))
{
+&& !DataflowRunner.hasExperiment(options,
"disable_per_worker_metrics")) {
enableBigQueryMetrics();
}
(beam) branch master updated: Fix typo (#31063)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 356311fb75d Fix typo (#31063)
356311fb75d is described below
commit 356311fb75dfdab574d4ff476d78bb246b5d842d
Author: Chamikara Jayalath
AuthorDate: Tue Apr 30 12:32:31 2024 -0700
Fix typo (#31063)
---
.../java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySinkMetrics.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySinkMetrics.java
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySinkMetrics.java
index ecdf8eee4bf..ac3ae16d00d 100644
---
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySinkMetrics.java
+++
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySinkMetrics.java
@@ -114,7 +114,7 @@ public class BigQuerySinkMetrics {
nameBuilder.addLabel(RPC_METHOD, method.toString());
MetricName metricName = nameBuilder.build(METRICS_NAMESPACE);
-// Create Exponential histogram buckets wtih the following parameters:
+// Create Exponential histogram buckets with the following parameters:
// 0 scale, resulting in bucket widths with a size 2 growth factor.
// 17 buckets, so the max latency of that can be stored is (2^17 millis ~=
130 seconds).
HistogramData.BucketType buckets = HistogramData.ExponentialBuckets.of(0,
17);
(beam) branch master updated: Elasticsearch enable Point In Time based searches (#30824)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 9612fe13eaf Elasticsearch enable Point In Time based searches (#30824)
9612fe13eaf is described below
commit 9612fe13eaf30eaa7b34c00026ec65895bd1a9fb
Author: pablo rodriguez defino
AuthorDate: Tue Apr 30 12:30:38 2024 -0700
Elasticsearch enable Point In Time based searches (#30824)
* first implementation for a PIT iterator on read PTransform
---
.../sdk/io/elasticsearch/ElasticsearchIOIT.java| 6 +
.../sdk/io/elasticsearch/ElasticsearchIOTest.java | 20 +-
.../elasticsearch/ElasticsearchIOTestCommon.java | 31 +-
.../io/elasticsearch/ElasticsearchIOTestUtils.java | 7 +-
.../beam/sdk/io/elasticsearch/ElasticsearchIO.java | 401 +
5 files changed, 391 insertions(+), 74 deletions(-)
diff --git
a/sdks/java/io/elasticsearch-tests/elasticsearch-tests-8/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOIT.java
b/sdks/java/io/elasticsearch-tests/elasticsearch-tests-8/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOIT.java
index 6801003e139..415af5e29a5 100644
---
a/sdks/java/io/elasticsearch-tests/elasticsearch-tests-8/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOIT.java
+++
b/sdks/java/io/elasticsearch-tests/elasticsearch-tests-8/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOIT.java
@@ -98,6 +98,12 @@ public class ElasticsearchIOIT {
elasticsearchIOTestCommon.testRead();
}
+ @Test
+ public void testReadPITVolume() throws Exception {
+elasticsearchIOTestCommon.setPipeline(pipeline);
+elasticsearchIOTestCommon.testReadPIT();
+ }
+
@Test
public void testWriteVolume() throws Exception {
// cannot share elasticsearchIOTestCommon because tests run in parallel.
diff --git
a/sdks/java/io/elasticsearch-tests/elasticsearch-tests-8/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTest.java
b/sdks/java/io/elasticsearch-tests/elasticsearch-tests-8/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTest.java
index 6bf96360d53..4cf02212bdc 100644
---
a/sdks/java/io/elasticsearch-tests/elasticsearch-tests-8/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTest.java
+++
b/sdks/java/io/elasticsearch-tests/elasticsearch-tests-8/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTest.java
@@ -104,13 +104,31 @@ public class ElasticsearchIOTest implements Serializable {
elasticsearchIOTestCommon.testRead();
}
+ @Test
+ public void testReadPIT() throws Exception {
+// need to create the index using the helper method (not create it at
first insertion)
+// for the indexSettings() to be run
+createIndex(elasticsearchIOTestCommon.restClient, getEsIndex());
+elasticsearchIOTestCommon.setPipeline(pipeline);
+elasticsearchIOTestCommon.testReadPIT();
+ }
+
@Test
public void testReadWithQueryString() throws Exception {
// need to create the index using the helper method (not create it at
first insertion)
// for the indexSettings() to be run
createIndex(elasticsearchIOTestCommon.restClient, getEsIndex());
elasticsearchIOTestCommon.setPipeline(pipeline);
-elasticsearchIOTestCommon.testRead();
+elasticsearchIOTestCommon.testReadWithQueryString();
+ }
+
+ @Test
+ public void testReadWithQueryStringAndPIT() throws Exception {
+// need to create the index using the helper method (not create it at
first insertion)
+// for the indexSettings() to be run
+createIndex(elasticsearchIOTestCommon.restClient, getEsIndex());
+elasticsearchIOTestCommon.setPipeline(pipeline);
+elasticsearchIOTestCommon.testReadWithQueryAndPIT();
}
@Test
diff --git
a/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTestCommon.java
b/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTestCommon.java
index d2e8efe4899..d1eeb610b65 100644
---
a/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTestCommon.java
+++
b/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTestCommon.java
@@ -235,17 +235,36 @@ class ElasticsearchIOTestCommon implements Serializable {
pipeline.run();
}
+ /** Point in Time search is currently available for Elasticsearch version
8+. */
+ void testReadPIT() throws Exception {
+if (!useAsITests) {
+ ElasticsearchIOTestUtils.insertTestDocuments(connectionConfiguration,
numDocs, restClient);
+}
+
+PCollection output =
+pipeline.apply(
+
(beam) branch master updated: remove atomic and wait from strimzi helm terrafrom (#31132)
This is an automated email from the ASF dual-hosted git repository.
damondouglas pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 5f71e6a592c remove atomic and wait from strimzi helm terrafrom (#31132)
5f71e6a592c is described below
commit 5f71e6a592ce10c7390f2bdb4f824c1bd59cc00e
Author: Vlado Djerek
AuthorDate: Tue Apr 30 17:30:56 2024 +0200
remove atomic and wait from strimzi helm terrafrom (#31132)
---
.test-infra/kafka/strimzi/01-strimzi-operator/kafka.tf | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/.test-infra/kafka/strimzi/01-strimzi-operator/kafka.tf
b/.test-infra/kafka/strimzi/01-strimzi-operator/kafka.tf
index a51158c1389..7a8ad1d5f24 100644
--- a/.test-infra/kafka/strimzi/01-strimzi-operator/kafka.tf
+++ b/.test-infra/kafka/strimzi/01-strimzi-operator/kafka.tf
@@ -24,8 +24,7 @@ resource "helm_release" "strimzi-helm-release" {
chart= var.chart_name
version = var.chart_version
- atomic = "true"
- timeout = 500
+ wait = false
set {
name = "watchAnyNamespace"
(beam) branch dependabot/gradle/com.gradle.common-custom-user-data-gradle-plugin-2.0.1 updated (d6690fae8ff -> 885958952da)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/com.gradle.common-custom-user-data-gradle-plugin-2.0.1 in repository https://gitbox.apache.org/repos/asf/beam.git discard d6690fae8ff Bump com.gradle.common-custom-user-data-gradle-plugin add bcb40cf4e4a Change caching of global window inputs to be guarded by experiment (#31013) add 2eb1a756258 [Python] Clean doc related to write data in bigquery.py (#30887) add 04ff4bdd7fc Support BQ clustering with value provider (#30460) add 61153bbda6a Update documentation of @SchemaFieldNumber (#30273) (#30277) add bb0b63cb940 Bump TPCDS test Flink version (#31041) add 70e067e1fde fix url for content security (#31043) add bb310e7e907 Change type for UnboundedReaderMaxReadTimeSec (#31037) add b69e8c615af Updates Python Dev container used by Dataflow (#31029) add 76c77cd28ae Fix typo in count_unique_words() (#31023) add 3e52e3554a0 Add code change guide contributor-doc (#30879) add d05196dea8a Upgrade the version of GRPC to pick up a fix for #30867 (#31044) add fd1003034ff Add ScaleToGaussian TFTransform (#31026) add 98b1f03125c Better and earlier error for missing service loader metadata. (#31042) add 40ec29ad289 fix website embedding issue (#31019) add 31f1c7968d4 fix url for content security add 93aa62c2900 Merge pull request #31058 from svetakvsundhar/broken_links add 49da7a34d99 Bump golang.org/x/net from 0.22.0 to 0.23.0 in /sdks (#31054) add 516ac97659f fix examples on website (#31066) add 1a5dc1c7fab Replace Source type for BigTable and Kafka stress tests (#30999) add 8dcbf967a1a [#30083][prism] Stabilize additional teststream cases. (#31046) add 2472695eb04 filter out outdated workflow runs (#31071) add f8b81211049 Fix test failure due to locale sensitivity. (#31069) add 718af0d7652 Don't use isolated builds when building an sdist for a pipeline package supplied in --setup_file option. (#31070) add 5a86b3fc922 Bump golang.org/x/net from 0.17.0 to 0.23.0 in /learning/katas/go (#31050) add 3dbbad98c80 Bump golang.org/x/net from 0.17.0 to 0.23.0 in /.test-infra/mock-apis (#31051) add a0dad088980 Bump golang.org/x/net from 0.22.0 to 0.24.0 in /sdks (#31065) add 37609ba70fa Managed Transform protos & translation; Iceberg SchemaTransforms & translation (#30910) add de665b77d9f [YAML] - Normalize YAML PubSub format (#31068) add ba7ea856ffe Revert "Change type for UnboundedReaderMaxReadTimeSec (#31037)" (#31082) add 58bea3f19e5 Fix building release candidate (#31084) add ab27f6f0358 Refactor Residuals to prepare for processing time scheduling. (#31057) add 1ebd5039a97 Simplify intermediate data in Iceberg sink; use manifest files add 485c5198384 Merge pull request #31086: Simplify intermediate data in Iceberg sink; use manifest files add 45e78572e8f python sdk: fix several bugs regarding avto <-> beam schema conversion (#30770) add 5cce8d79925 Add usage counters to ListView side inputs. add 83a90f2bbb4 Merge pull request #31083 Add usage counters to ListView side inputs. add 495e9b2eb7e Python package string change (#31093) add 8454cc95030 Change assertion and timeout for kafka stress test (#31089) add 2db9b8097b7 Mention known issues in release notes. (#31088) add e7847998a44 use strimzi kafka on beam_utility cluster (#30967) add 0f69dc19a5d remove cloudsdk environment variable, not needed here (#31102) add 3d3669e2bbc Use java 11 for java_tests workflow (#31101) add 3c8a8815c00 Add ability to load multiple copies of a model across processes (#31052) add 088c854d7a3 Fix :examples:java:twitter:execute task (#31103) add b33a8438ad3 Revert global snake_case convention for SchemaTransforms (#31109) add 15f7f02a79d Add UnboundedReaderMaxReadTimeMs to DataflowPipelineDebugOptions, deprecate UnboundedReaderMaxReadTimeSec (#31091) add 5c3786e3eb0 Try fixing build wheels (#31115) add 3e5a658498e fix test added in previous commit (#31116) add c20e329a326 Fix reporting metrics not supported warning for BigQueryIO Direct read (#31096) add 3329edba79f [Python] Add feast feature store handler for enrichment transform (#30957) add 68f6b551541 One more macos change (#31123) add 28a2682d54c add terraform for utility cluster. Add name override to gke (#30847) add 673da546c14 [#30083][prism] Factor out hold tracking to dedicated structures (#31105) add 08fc33b26cc Touch trigger files (add some comments as well) add 7ade06f6f77 Add Redistribute to Beam model protos add f49ac7d10a0 Add Redistribute transform to Java SDK add 18ebc454928 Add direct Dataflow translation of Redistribute transform add 21e3fa1eedf Add Redistribute translation to Spark runner add 8f1d3da4399 Add Redistribute translation
(beam) branch master updated (413af128937 -> e3b4d41f6b8)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 413af128937 Allow users to configure wait options for new Neo4j databases (#31129) add 22535521e7b Bump com.gradle.enterprise from 3.15.1 to 3.17.2 new e3b4d41f6b8 Merge pull request #31032: Bump com.gradle.enterprise from 3.15.1 to 3.17.2 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: settings.gradle.kts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(beam) branch dependabot/gradle/com.gradle.enterprise-3.17.2 deleted (was 22535521e7b)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/com.gradle.enterprise-3.17.2 in repository https://gitbox.apache.org/repos/asf/beam.git was 22535521e7b Bump com.gradle.enterprise from 3.15.1 to 3.17.2 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) 01/01: Merge pull request #31032: Bump com.gradle.enterprise from 3.15.1 to 3.17.2
This is an automated email from the ASF dual-hosted git repository. kenn pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit e3b4d41f6b8a7a36891a268c5ee513e70fe5530c Merge: 413af128937 22535521e7b Author: Kenneth Knowles AuthorDate: Tue Apr 30 10:40:11 2024 -0400 Merge pull request #31032: Bump com.gradle.enterprise from 3.15.1 to 3.17.2 settings.gradle.kts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(beam) branch master updated: Allow users to configure wait options for new Neo4j databases (#31129)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 413af128937 Allow users to configure wait options for new Neo4j
databases (#31129)
413af128937 is described below
commit 413af1289373079319ba77b3f233751fbdfc3bc9
Author: Florent Biville <445792+fbivi...@users.noreply.github.com>
AuthorDate: Tue Apr 30 16:23:14 2024 +0200
Allow users to configure wait options for new Neo4j databases (#31129)
* Allow users to configure wait options for new Neo4j databases
Since v5 of Neo4j, databases are created in an asynchronous manner.
Before this commit, `Neo4jResourceManager` did not allow users
to specify whether to wait or not for the database creation (via
the corresponding Cypher syntax element).
This commit fixes that, while keeping the behavior the same as
before, by default (which is to not wait).
* Format
* Fold back intermediate classes
* Format again
---
.../apache/beam/it/neo4j/DatabaseWaitOption.java | 20 +++
.../apache/beam/it/neo4j/DatabaseWaitOptions.java | 68 ++
.../apache/beam/it/neo4j/Neo4jResourceManager.java | 41 +++--
.../beam/it/neo4j/Neo4jResourceManagerIT.java | 2 +-
.../beam/it/neo4j/Neo4jResourceManagerTest.java| 15 -
5 files changed, 138 insertions(+), 8 deletions(-)
diff --git
a/it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOption.java
b/it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOption.java
new file mode 100644
index 000..5feabdced72
--- /dev/null
+++ b/it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOption.java
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.it.neo4j;
+
+public interface DatabaseWaitOption {}
diff --git
a/it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOptions.java
b/it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOptions.java
new file mode 100644
index 000..d240d024c0a
--- /dev/null
+++ b/it/neo4j/src/main/java/org/apache/beam/it/neo4j/DatabaseWaitOptions.java
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.it.neo4j;
+
+public class DatabaseWaitOptions {
+
+ public static DatabaseWaitOption waitDatabase() {
+return DatabaseWait.WAIT;
+ }
+
+ public static DatabaseWaitOption waitDatabase(int seconds) {
+return new DatabaseWaitInSeconds(seconds);
+ }
+
+ public static DatabaseWaitOption noWaitDatabase() {
+return DatabaseNoWait.NO_WAIT;
+ }
+
+ static String asCypher(DatabaseWaitOption option) {
+if (option == null || option == DatabaseNoWait.NO_WAIT) {
+ return "NOWAIT";
+}
+if (option == DatabaseWait.WAIT) {
+ return "WAIT";
+}
+if (option instanceof DatabaseWaitInSeconds) {
+ DatabaseWaitInSeconds wait = (DatabaseWaitInSeconds) option;
+ return String.format("WAIT %s SECONDS", wait.getSeconds());
+}
+throw new Neo4jResourceManagerException(
+String.format("Unsupported wait option type %s", option.getClass()));
+ }
+
+ private enum DatabaseNoWait implements DatabaseWaitOption {
+NO_WAIT;
+ }
+
+ private enum DatabaseWait implements DatabaseWaitOption {
+WAIT;
+ }
+
+ private
(beam) branch master updated: add finalizers to kafka loadbalancer services (#31130)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new f0d06051072 add finalizers to kafka loadbalancer services (#31130) f0d06051072 is described below commit f0d06051072da5b64c496dc8456e56bab9aca71b Author: Vlado Djerek AuthorDate: Tue Apr 30 14:49:29 2024 +0200 add finalizers to kafka loadbalancer services (#31130) --- .../overlays/gke-internal-load-balanced/listeners.yaml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.test-infra/kafka/strimzi/02-kafka-persistent/overlays/gke-internal-load-balanced/listeners.yaml b/.test-infra/kafka/strimzi/02-kafka-persistent/overlays/gke-internal-load-balanced/listeners.yaml index de9343de687..da708591525 100644 --- a/.test-infra/kafka/strimzi/02-kafka-persistent/overlays/gke-internal-load-balanced/listeners.yaml +++ b/.test-infra/kafka/strimzi/02-kafka-persistent/overlays/gke-internal-load-balanced/listeners.yaml @@ -37,6 +37,9 @@ spec: type: loadbalancer tls: false configuration: +finalizers: +- gke.networking.io/l4-ilb-v1 +- service.kubernetes.io/load-balancer-cleanup bootstrap: annotations: cloud.google.com/load-balancer-type: "Internal"
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 654d17cd242 Updating config from bot
654d17cd242 is described below
commit 654d17cd2425ee582d4d7b1d130775c44deb1e55
Author: github-actions
AuthorDate: Tue Apr 30 12:07:31 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31133.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31133.json
b/scripts/ci/pr-bot/state/pr-state/pr-31133.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31133.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8a90b50a88e Updating config from bot 8a90b50a88e is described below commit 8a90b50a88eb34b0d3972bf5c1f75fb43dc180cf Author: github-actions AuthorDate: Tue Apr 30 11:52:13 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-31130.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31130.json b/scripts/ci/pr-bot/state/pr-state/pr-31130.json index 26e4be5dc8a..24ff5131212 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-31130.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-31130.json @@ -4,7 +4,7 @@ "no-matching-label": "chamikaramj" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new d4ff482855f Updating config from bot
d4ff482855f is described below
commit d4ff482855fcecfa5f3166cd61e94e383bbd7792
Author: github-actions
AuthorDate: Tue Apr 30 09:11:51 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31132.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31132.json
b/scripts/ci/pr-bot/state/pr-state/pr-31132.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31132.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new dd72b91b5cf Updating config from bot dd72b91b5cf is described below commit dd72b91b5cf2bf6c0266d6806b0281a49606921f Author: github-actions AuthorDate: Tue Apr 30 08:47:50 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-31062.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31062.json b/scripts/ci/pr-bot/state/pr-state/pr-31062.json index b2b2cecc92a..a00a5377c10 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-31062.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-31062.json @@ -5,7 +5,7 @@ "build": "Abacn" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 450c3656283 Updating config from bot
450c3656283 is described below
commit 450c3656283bb5a6ee08525a7682f306fc1b9f12
Author: github-actions
AuthorDate: Tue Apr 30 08:35:07 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-31062.json | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31062.json
b/scripts/ci/pr-bot/state/pr-state/pr-31062.json
index 242a48d7d3b..b2b2cecc92a 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-31062.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31062.json
@@ -1,7 +1,10 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "jrmccluskey",
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1e1e60e8ad1 Updating config from bot
1e1e60e8ad1 is described below
commit 1e1e60e8ad1bf19730d90412264ca922d09d53cb
Author: github-actions
AuthorDate: Tue Apr 30 08:35:10 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 27506c265cb..6d62c141f88 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1714135257527,
-"Abacn": 1713980068872
+"Abacn": 1714466105691
}
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 7fb975ca1ea Updating config from bot
7fb975ca1ea is described below
commit 7fb975ca1ea362c032ab70fdce72e833e2223a5f
Author: github-actions
AuthorDate: Tue Apr 30 08:35:08 2024 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 944c2113e88..3d5903824ae 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
"label": "go",
"dateOfLastReviewAssignment": {
"lostluck": 1713764066934,
-"jrmccluskey": 1713528829587,
+"jrmccluskey": 1714466105691,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
"riteshghorse": 1713535545036
