This is an automated email from the ASF dual-hosted git repository.
pabloem pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 8ccd1d70c04 including GitHub Actions to schedule daily audits and Add
documentati… (#39116)
8ccd1d70c04 is described below
commit 8ccd1d70c044016d1ec63e825e972dae1cfb0001
Author: HansMarcus01 <[email protected]>
AuthorDate: Fri Jun 26 12:02:55 2026 -0600
including GitHub Actions to schedule daily audits and Add documentati…
(#39116)
* including GitHub Actions to schedule daily audits and Add documentation
for the changes made to manage keys created outside of the Google Cloud Secret
Manager key rotation system
* correcting a typo in the file name
* Fix PreCommit GHA and PreCommit Whitespace, and preparing the test of the
action from the pull request
* Testing the GitHub action from the PR
* Remove the pull request configuration from the github Action
---
.github/workflows/README.md | 1 +
.../beam_Infrastructure_AuditUnmanagedKeys.yml | 69 ++++++++++++++++++++++
infra/enforcement/README.md | 12 +++-
3 files changed, 79 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/README.md b/.github/workflows/README.md
index 4a32aa271df..f6b1a440e17 100644
--- a/.github/workflows/README.md
+++ b/.github/workflows/README.md
@@ -545,3 +545,4 @@ PostCommit Jobs run in a schedule against master branch and
generally do not get
| [ Infrastructure Policy Enforcer
](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml)
| N/A |
[](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml?query=event%3Aschedule)
|
| [ Modify the GCP User Roles according to the infra/users.yml file
](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml)
| N/A |
[](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml?query=event%3Aschedule)
|
| [ Service Account Keys Management
](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml)
| N/A |
[](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml?query=event%3Aschedule)
|
+| [ Unmanaged Service Accounts Keys Audit
](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml)
| N/A |
[](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml?query=event%3Aschedule)
|
diff --git a/.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml
b/.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml
new file mode 100644
index 00000000000..6d34bd1d156
--- /dev/null
+++ b/.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml
@@ -0,0 +1,69 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This workflow works with the GCP security log analyzer to
+# generate weekly security reports and initialize log sinks
+
+name: Unmanaged Service Accounts Keys Audit
+
+on:
+ workflow_dispatch:
+ schedule:
+ # Every day at 00:00 UTC
+ - cron: '0 0 * * *'
+
+concurrency:
+ group: ${{ github.workflow }}
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+ issues: write
+ id-token: write
+
+jobs:
+ beam_UnmanagedKeysAudit:
+ name: Audit Unmanaged Service Account Keys
+ runs-on: [self-hosted, ubuntu-24.04, main]
+ timeout-minutes: 30
+ steps:
+ - uses: actions/checkout@v6
+
+ - name: Setup gcloud
+ uses:
google-github-actions/setup-gcloud@aa5489c8933f4cc7a4f7d45035b3b1440c9c10db
+
+ - name: Setup Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.13'
+
+ - name: Install Python dependencies
+ working-directory: ./infra/enforcement
+ run: |
+ python -m pip install --upgrade pip
+ pip install -r requirements.txt
+
+ - name: Run Unmanaged Service Account Keys Audit
+ working-directory: ./infra/enforcement
+ env:
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ GITHUB_REPOSITORY: ${{ github.repository }}
+ run: python account_keys.py --action announce
+
+
+
+
diff --git a/infra/enforcement/README.md b/infra/enforcement/README.md
index 8136081fed7..6d883f7e680 100644
--- a/infra/enforcement/README.md
+++ b/infra/enforcement/README.md
@@ -134,8 +134,9 @@ The enforcement tools are integrated with GitHub Actions to
provide automated co
### Workflow Configuration
-The GitHub Actions workflow
(`.github/workflows/beam_Infrastructure_PolicyEnforcer.yml`) runs:
-- **Schedule**: Weekly on Mondays at 9:00 AM UTC
+The repository includes workflows for different security domains:
+- **IAM Policy Enforcer**
(`.github/workflows/beam_Infrastructure_PolicyEnforcer.yml`): Runs weekly on
Mondays at 9:00 AM UTC.
+- **Unmanaged Keys Audit**
(`.github/workflows/beam_Infrastructure_AuditUnmanagedKeys.yml`): Runs daily at
00:00 UTC. It manages the continuous execution of the `account_keys.py` script
to swiftly detect rogue service account keys generated outside the official
rotation system.
- **Manual trigger**: Can be triggered manually via `workflow_dispatch`
- **Actions**: Runs both IAM and Account Keys enforcement with the `announce`
action
@@ -170,7 +171,9 @@ python account_keys.py --action generate
### Actions
- **check**: Validates service account keys and their permissions against
defined policies and reports any differences (default behavior)
-- **announce**: Creates or updates a GitHub issue and sends an email
notification when service account keys policies differ from the defined ones.
If no open issue exists, creates a new one; if an open issue exists, updates
the issue body with current violations
+- **announce**: Creates or updates a GitHub issue and sends an email
notification when service account keys policies differ from the defined ones.
+ - For general configuration errors, it updates the main compliance issue.
+ - **For unmanaged/rogue keys (Security Alerts)**, it consolidates alerts
into a dedicated `[SECURITY]` issue acting as a live dashboard. It updates the
issue by placing the newest audit report at the top and moving the previous
reports into a collapsed `<details>` history section. If the keys are revoked
and the infrastructure becomes healthy, the system automatically resolves and
closes the issue.
- **print**: Prints announcement details for testing purposes without creating
actual GitHub issues or sending emails
- **generate**: Updates the compliance file to match the current GCP service
account keys and Secret Manager permissions
@@ -183,6 +186,9 @@ The Account Keys enforcement tool provides the following
capabilities:
- **Permission Validation**: Ensures that Secret Manager permissions match the
declared authorized users
- **Compliance Reporting**: Identifies missing service accounts, undeclared
managed secrets, and permission mismatches
- **Automatic Remediation**: Can automatically update the compliance file to
match current infrastructure state
+- **Unmanaged Key Detection**: Identifies physical keys in IAM that were
created outside the automated rotation system by comparing them against the
legal, managed keys registered in Google Cloud Secret Manager.
+- **Stateful Security Alerts**: Consolidates security violations into a
single, dynamically updated GitHub issue, keeping a collapsed `<details>`
history of past scans to prevent alert fatigue.
+- **Self-Healing Resolution**: Automatically closes the security GitHub issue
when all unmanaged keys have been successfully revoked.
### Configuration
